[PATCH RFC 00/13] drm/powervr_sgx: Add a DRM driver for PowerVR SGX5xx GPUs

[PATCH RFC 00/13] drm/powervr_sgx: Add a DRM driver for PowerVR SGX5xx GPUs

[Joseph Telaak]

This RFC asks for design review of a render-only DRM driver for the
PowerVR SGX5xx (Series5) GPUs. It is not for merging: the Mesa driver
isn't in Mesa yet, the firmware isn't in linux-firmware, and until each
client has its own parameter buffer it isn't safe to expose to
untrusted clients.

The first platform is the SGX545 of the Intel Atom CE5300 (PCI
8086:089b). The second is Cedarview, where gma500 publishes the SGX as
an auxiliary device (patches 2 and 3). The OMAP3630's SGX530 is there
for bring-up only: it has one memory context, so the driver refuses it
without powervr_sgx.allow_unisolated=1, which drops client isolation and
isn't for general use.

Design
- drm_sched, one job on the GPU at a time; drm_gpuvm, a page directory
  per VM; GEM shmem; explicit sync only. A core table; an unknown core
  is refused before any register write.
- An open microkernel (MIT OR Apache-2.0) [4], loaded only in secure
  mode, where no client maps the firmware's memory.
- Commands are plain structs, checked field by field; uploaded shader
  code is checked instruction by instruction.
- The GPU doesn't snoop: a job lists the imported ranges it touches,
  and the kernel flushes them around it.

Known limits, to fix before a v1:
- One parameter buffer serves every client, so one client's GPU work
  can read and write another's binned geometry.
- Kernel objects and page tables use CPU physical addresses (x86
  without an IOMMU), imports keep the exporter's vmap, and cache flushes
  run uncapped in the fence-signaling path. All three move to the DMA
  API.
- Some firmware words are still written by index or named unk_*.
- Transfer and NULL jobs have no Mesa user, and a client's first
  transfer can fault after another client's transfers.

Userspace is a Gallium driver with a NIR compiler [3]. On this tree, a
2,146-case sample of dEQP-GLES2 on the CE5300 gives the same results as
before the last uAPI revision (2,082 pass, 7 fail, 56 not supported, 1
warning), with no GPU resets; it is a sample, not the full suite.

Questions
1. Imagination said in 2023 that SGX should have its own driver [1].
   Alessio, Luigi: beside drivers/gpu/drm/imagination, or under it?
2. gma500 publishes the SGX as an auxiliary device, as nova-core does
   for nova-drm, and forwards its interrupt through a callback, because
   CE5300 kernels have no gma500 to link against. Gabriel Gama is
   preparing gma500's SGX535 code [2]. Is a separate driver the right
   split?
3. Is cache maintenance in SUBMIT_JOBS, as per-job ranges, acceptable,
   or should it be a BO_SYNC-style ioctl?
4. On x86, shmem flips every map_wc object's pages to write-combined
   and back. Is TTM, cached pages with flushes, or a GFP_WC allocator
   the way?

I maintain this alone for now and would welcome a reviewer.

Most of this series was written using the assistance of an AI coding
assistant, Claude Code (Anthropic), working under my direction over many
sessions: the kernel driver, the open microkernel, the Mesa driver and
most of the documentation. I set the design and the uAPI decisions,
chose what to fix or disclose, and reviewed the changes; before posting,
the series also had an independent automated review whose findings are
fixed or listed above. All the testing described here ran on my own
hardware. I'm responsible for every line and will answer review
questions myself.

Testing: on the CE5300, 7.3.3 kernels with this series plus board
patches outside it: the uAPI, frame, hang, PRIME and unbind tests [5]
and a three-client soak, also under lockdep, KASAN, UBSAN, kmemleak and
DMA-API debugging. Testers ran an earlier version of the import path on
four Cedarview machines; this posting's path isn't retested there.
Suspend is tested with pm_test only. CI applies each patch to
drm-misc-next and runs checkpatch --strict and kernel-doc on it, and gcc
and clang W=1, sparse and smatch on what it builds (the driver builds
from patch 12).

[1] https://lore.kernel.org/r/ccfae572cb6a5ba70eea273e641b66cd7f6fe58b.camel@imgtec.com
[2] https://lore.kernel.org/r/20261109063303.32658-1-operatingsystemsdepression@gmail.com
[3] https://github.com/The1TrueJoe/powervr_sgx/tree/525bdadb0e088bd3f64f0407b8cb0a16c6c03a2d/mesa
[4] https://github.com/The1TrueJoe/powervr_sgx/tree/525bdadb0e088bd3f64f0407b8cb0a16c6c03a2d/ukernel
[5] https://github.com/The1TrueJoe/powervr_sgx/tree/525bdadb0e088bd3f64f0407b8cb0a16c6c03a2d/drm/tests

---
Joseph Telaak (13):
      drm/gem-shmem: Map pages for the DMA device
      drm/gma500: Publish the Cedarview SGX as an auxiliary device
      drm/gma500: Export GEM objects as dma-bufs
      drm/powervr_sgx: Add the uAPI header and its documentation
      drm/powervr_sgx: Add the core table, registers and device bring-up
      drm/powervr_sgx: Load and start the microkernel
      drm/powervr_sgx: Add GEM objects, VMs and dma-buf sharing
      drm/powervr_sgx: Add contexts, the scheduler and SUBMIT_JOBS
      drm/powervr_sgx: Add the client command checks
      drm/powervr_sgx: Add render jobs
      drm/powervr_sgx: Validate shader code in the kernel
      drm/powervr_sgx: Add the DRM driver, power management and debugging
      drm/powervr_sgx: Add KUnit tests

 Documentation/gpu/drivers.rst                      |    1 +
 Documentation/gpu/powervr_sgx/index.rst            |  201 ++++
 Documentation/gpu/powervr_sgx/uapi.rst             |  133 +++
 MAINTAINERS                                        |   11 +
 drivers/gpu/drm/Kconfig                            |    1 +
 drivers/gpu/drm/Makefile                           |    1 +
 drivers/gpu/drm/drm_gem_shmem_helper.c             |    8 +-
 drivers/gpu/drm/gma500/Kconfig                     |    1 +
 drivers/gpu/drm/gma500/Makefile                    |    3 +-
 drivers/gpu/drm/gma500/cdv_device.c                |    5 +
 drivers/gpu/drm/gma500/gem.c                       |  192 +++-
 drivers/gpu/drm/gma500/gem.h                       |    1 +
 drivers/gpu/drm/gma500/psb_drv.c                   |   42 +-
 drivers/gpu/drm/gma500/psb_drv.h                   |   26 +
 drivers/gpu/drm/gma500/psb_irq.c                   |   32 +-
 drivers/gpu/drm/gma500/sgx_aux.c                   |  102 ++
 drivers/gpu/drm/powervr_sgx/Kconfig                |   65 ++
 drivers/gpu/drm/powervr_sgx/Makefile               |   34 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_aux.c           |   83 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cache.h         |   66 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c           |  208 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.h           |   52 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.c           |  234 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.h           |  132 +++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.c    |   62 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.h    |   13 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.c      |  353 ++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.h      |   43 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_core.c          |  300 +++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_core.h          |  196 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.c       |   78 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.h       |   13 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_device.c        |  461 ++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_device.h        |  354 ++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_drv.c           |  378 +++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_drv.h           |   52 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_dump.c          |  227 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_dump.h          |   16 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c            |  736 ++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_fw.h            |   77 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_fwif.h          |  519 +++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_gem.c           |  120 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_gem.h           |   46 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_hw.h            |   55 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_job.c           | 1028 +++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_job.h           |  143 +++
 drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.c          |  285 +++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.h          |   82 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.c           |  281 +++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.h           |   28 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_of.c            |  119 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_pds.c           |   97 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_pds.h           |   37 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_pm.c            |  138 +++
 drivers/gpu/drm/powervr_sgx/pvrsgx_pm.h            |   30 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c         |  207 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_prime.h         |   26 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_regs.h          |   79 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_render.c        |  957 ++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_render.h        |  122 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_trace.c         |    5 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_trace.h         |  134 +++
 drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.c      |  175 +++
 drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.h      |   38 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c          |  240 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_usse.h          |   45 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_usse_table.h    |  342 ++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c      |  192 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_validate.h      |   25 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_vm.c            |  799 +++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_vm.h            |   69 ++
 drivers/gpu/drm/powervr_sgx/tests/.kunitconfig     |    5 +
 .../gpu/drm/powervr_sgx/tests/pvrsgx_ccb_test.c    |  111 ++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_core_test.c   |  119 ++
 .../drm/powervr_sgx/tests/pvrsgx_equiv_golden.h    |   32 +
 .../gpu/drm/powervr_sgx/tests/pvrsgx_equiv_test.c  |  555 +++++++++
 drivers/gpu/drm/powervr_sgx/tests/pvrsgx_fw_test.c |  168 +++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_kobj_test.c   |  203 ++++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_prime_test.c  |  125 +++
 drivers/gpu/drm/powervr_sgx/tests/pvrsgx_rt_test.c |   63 ++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_usse_test.c   |  145 +++
 .../drm/powervr_sgx/tests/pvrsgx_validate_test.c   |   32 +
 include/drm/gma500_sgx.h                           |   52 +
 include/uapi/drm/powervr_sgx_drm.h                 | 1175 ++++++++++++++++++++
 84 files changed, 14178 insertions(+), 63 deletions(-)
---
base-commit: c88084027d0e922dd925b96e5a0d184ba04a8f43
change-id: 20261112-powervr-sgx-c1b67d5efaa2

Best regards,
--  
Joseph Telaak <joseph@telaak.dev>

[PATCH RFC 01/13] drm/gem-shmem: Map pages for the DMA device

[Joseph Telaak]

A DRM device whose DMA goes through another device sets it with
drm_dev_set_dma_dev(). The DMA helper allocates and maps for that
device, and PRIME imports attach with it, but the shmem helper still
maps and unmaps its pages for drm_device.dev, and
drm_gem_shmem_prime_import_no_map() still attaches with it. On such a
device that is a struct device without DMA ops or a DMA mask.

Map and unmap shmem pages, and attach in
drm_gem_shmem_prime_import_no_map(), for drm_dev_dma_dev(), as the DMA
helper and drm_prime do. Nothing changes for drivers that don't set a
DMA device. Of those that do and use the shmem helpers (gud, udl,
gm12u320, appletbdrm), none maps an object's pages through
drm_gem_shmem_get_pages_sgt() or imports with
drm_gem_shmem_prime_import_no_map().

The first user is powervr_sgx, later in this series: behind gma500 its
DRM device is an auxiliary device, and its DMA goes through the GPU's
PCI function.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/drm_gem_shmem_helper.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c
index d654a22a3..4bce0a716 100644
--- a/drivers/gpu/drm/drm_gem_shmem_helper.c
+++ b/drivers/gpu/drm/drm_gem_shmem_helper.c
@@ -175,7 +175,7 @@ void __drm_gem_shmem_free_sgt_locked(struct drm_gem_shmem_object *shmem)
 {
 	dma_resv_assert_held(shmem->base.resv);
 
-	dma_unmap_sgtable(shmem->base.dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
+	dma_unmap_sgtable(drm_dev_dma_dev(shmem->base.dev), shmem->sgt, DMA_BIDIRECTIONAL, 0);
 	sg_free_table(shmem->sgt);
 	kfree(shmem->sgt);
 	shmem->sgt = NULL;
@@ -535,7 +535,7 @@ void drm_gem_shmem_purge_locked(struct drm_gem_shmem_object *shmem)
 
 	drm_WARN_ON(obj->dev, !drm_gem_shmem_is_purgeable(shmem));
 
-	dma_unmap_sgtable(dev->dev, shmem->sgt, DMA_BIDIRECTIONAL, 0);
+	dma_unmap_sgtable(drm_dev_dma_dev(dev), shmem->sgt, DMA_BIDIRECTIONAL, 0);
 	sg_free_table(shmem->sgt);
 	kfree(shmem->sgt);
 	shmem->sgt = NULL;
@@ -854,7 +854,7 @@ static struct sg_table *drm_gem_shmem_get_pages_sgt_locked(struct drm_gem_shmem_
 		goto err_put_pages;
 	}
 	/* Map the pages for use by the h/w. */
-	ret = dma_map_sgtable(obj->dev->dev, sgt, DMA_BIDIRECTIONAL, 0);
+	ret = dma_map_sgtable(drm_dev_dma_dev(obj->dev), sgt, DMA_BIDIRECTIONAL, 0);
 	if (ret)
 		goto err_free_sgt;
 
@@ -964,7 +964,7 @@ struct drm_gem_object *drm_gem_shmem_prime_import_no_map(struct drm_device *dev,
 		return obj;
 	}
 
-	attach = dma_buf_attach(dma_buf, dev->dev);
+	attach = dma_buf_attach(dma_buf, drm_dev_dma_dev(dev));
 	if (IS_ERR(attach))
 		return ERR_CAST(attach);
 

-- 
2.50.1

[PATCH RFC 02/13] drm/gma500: Publish the Cedarview SGX as an auxiliary device

[Joseph Telaak]

The GMA3600/3650 (Cedarview) GPU is a PowerVR SGX545 behind the same PCI
function and register BAR as the display engine gma500 drives. gma500
uses the SGX for nothing but an MMU it sets up at probe and fills when it
pins framebuffers; no gma500 code reads through that MMU. So give the SGX
to a driver of its own: publish an auxiliary device, "gma500_gfx.sgx",
carrying the mapped SGX register window, which SGX core it is, and a hook
that routes the SGX's interrupt to that driver.

Each chip's psb_ops says whether its SGX has a driver of its own, and as
which core. On such a chip gma500 hands the SGX over for good: it doesn't
reset it or program its BIF at probe, creates no SGX MMU and maps nothing
into one, masks the SGX's events at interrupt (re)install until a
handler is registered, and passes the SGX's interrupt to the registered
handler. Teardown clears the handler itself, whatever the SGX driver did.

The interrupt reaches the SGX driver through a callback on the auxiliary
device, not an exported symbol: an exported gma500 symbol would make
that driver depend on gma500, and the same driver runs the Intel Atom
CE5300, where the SGX is a PCI function of its own and there is no
gma500.

The auxiliary device's parent is the GPU's PCI device, so runtime PM on
the SGX side would keep gma500, and with it the graphics power island,
awake; gma500 holds a runtime-PM reference for its whole life today.

Only Cedarview's psb_ops names an SGX. Poulsbo and Oaktrail leave it NULL
and are unchanged until an SGX driver has run on them.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 MAINTAINERS                         |   1 +
 drivers/gpu/drm/gma500/Kconfig      |   1 +
 drivers/gpu/drm/gma500/Makefile     |   3 +-
 drivers/gpu/drm/gma500/cdv_device.c |   5 ++
 drivers/gpu/drm/gma500/gem.c        |  12 +++--
 drivers/gpu/drm/gma500/psb_drv.c    |  42 +++++++++------
 drivers/gpu/drm/gma500/psb_drv.h    |  26 +++++++++
 drivers/gpu/drm/gma500/psb_irq.c    |  32 +++++++----
 drivers/gpu/drm/gma500/sgx_aux.c    | 102 ++++++++++++++++++++++++++++++++++++
 include/drm/gma500_sgx.h            |  52 ++++++++++++++++++
 10 files changed, 246 insertions(+), 30 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index c4f4e4877..e1b0a4245 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9036,6 +9036,7 @@ L:	dri-devel@lists.freedesktop.org
 S:	Maintained
 T:	git https://gitlab.freedesktop.org/drm/misc/kernel.git
 F:	drivers/gpu/drm/gma500/
+F:	include/drm/gma500_sgx.h
 
 DRM DRIVERS FOR HISILICON HIBMC
 M:	Yongbang Shi <shiyongbang@huawei.com>
diff --git a/drivers/gpu/drm/gma500/Kconfig b/drivers/gpu/drm/gma500/Kconfig
index dc4c56638..bece965b2 100644
--- a/drivers/gpu/drm/gma500/Kconfig
+++ b/drivers/gpu/drm/gma500/Kconfig
@@ -4,6 +4,7 @@ config DRM_GMA500
 	depends on DRM && PCI && X86 && HAS_IOPORT
 	select DRM_CLIENT_SELECTION
 	select DRM_KMS_HELPER
+	select AUXILIARY_BUS
 	select FB_IOMEM_HELPERS if DRM_FBDEV_EMULATION
 	select I2C
 	select I2C_ALGOBIT
diff --git a/drivers/gpu/drm/gma500/Makefile b/drivers/gpu/drm/gma500/Makefile
index 58fed80c7..a933940e2 100644
--- a/drivers/gpu/drm/gma500/Makefile
+++ b/drivers/gpu/drm/gma500/Makefile
@@ -34,7 +34,8 @@ gma500_gfx-y += \
 	  psb_intel_lvds.o \
 	  psb_intel_modes.o \
 	  psb_intel_sdvo.o \
-	  psb_irq.o
+	  psb_irq.o \
+	  sgx_aux.o
 
 gma500_gfx-$(CONFIG_ACPI) +=  opregion.o
 gma500_gfx-$(CONFIG_DRM_FBDEV_EMULATION) +=  fbdev.o
diff --git a/drivers/gpu/drm/gma500/cdv_device.c b/drivers/gpu/drm/gma500/cdv_device.c
index fd6ea8998..f0a835366 100644
--- a/drivers/gpu/drm/gma500/cdv_device.c
+++ b/drivers/gpu/drm/gma500/cdv_device.c
@@ -563,6 +563,10 @@ static int cdv_chip_setup(struct drm_device *dev)
 
 /* CDV is much like Poulsbo but has MID like SGX offsets and PM */
 
+static const struct gma_sgx_info cdv_sgx_info = {
+	.core = 545,
+};
+
 const struct psb_ops cdv_chip_ops = {
 	.name = "GMA3600/3650",
 	.pipes = 2,
@@ -572,6 +576,7 @@ const struct psb_ops cdv_chip_ops = {
 	.sdvo_mask = (1 << 0),
 	.cursor_needs_phys = 0,
 	.sgx_offset = MRST_SGX_OFFSET,
+	.sgx = &cdv_sgx_info,
 	.chip_setup = cdv_chip_setup,
 	.errata = cdv_errata,
 
diff --git a/drivers/gpu/drm/gma500/gem.c b/drivers/gpu/drm/gma500/gem.c
index 2708e8c68..4a4ecfb7b 100644
--- a/drivers/gpu/drm/gma500/gem.c
+++ b/drivers/gpu/drm/gma500/gem.c
@@ -54,9 +54,10 @@ int psb_gem_pin(struct psb_gem_object *pobj)
 	set_pages_array_wc(pages, npages);
 
 	psb_gtt_insert_pages(dev_priv, &pobj->resource, pages);
-	psb_mmu_insert_pages(psb_mmu_get_default_pd(dev_priv->mmu), pages,
-			     (gpu_base + pobj->offset), npages, 0, 0,
-			     PSB_MMU_CACHED_MEMORY);
+	if (dev_priv->mmu)
+		psb_mmu_insert_pages(psb_mmu_get_default_pd(dev_priv->mmu), pages,
+				     (gpu_base + pobj->offset), npages, 0, 0,
+				     PSB_MMU_CACHED_MEMORY);
 
 	pobj->pages = pages;
 
@@ -93,8 +94,9 @@ void psb_gem_unpin(struct psb_gem_object *pobj)
 
 	npages = obj->size / PAGE_SIZE;
 
-	psb_mmu_remove_pages(psb_mmu_get_default_pd(dev_priv->mmu),
-			     (gpu_base + pobj->offset), npages, 0, 0);
+	if (dev_priv->mmu)
+		psb_mmu_remove_pages(psb_mmu_get_default_pd(dev_priv->mmu),
+				     (gpu_base + pobj->offset), npages, 0, 0);
 	psb_gtt_remove_pages(dev_priv, &pobj->resource);
 
 	/* Reset caching flags */
diff --git a/drivers/gpu/drm/gma500/psb_drv.c b/drivers/gpu/drm/gma500/psb_drv.c
index d17cb5b4a..c29cc64cc 100644
--- a/drivers/gpu/drm/gma500/psb_drv.c
+++ b/drivers/gpu/drm/gma500/psb_drv.c
@@ -145,6 +145,10 @@ static int psb_do_init(struct drm_device *dev)
 
 	spin_lock_init(&dev_priv->irqmask_lock);
 
+	/* The SGX's own driver sets it up. */
+	if (gma_sgx_external(dev_priv))
+		return 0;
+
 	PSB_WSGX32(0x00000000, PSB_CR_BIF_BANK0);
 	PSB_WSGX32(0x00000000, PSB_CR_BIF_BANK1);
 	PSB_RSGX32(PSB_CR_BIF_BANK1);
@@ -172,6 +176,7 @@ static void psb_driver_unload(struct drm_device *dev)
 	gma_backlight_exit(dev);
 	psb_modeset_cleanup(dev);
 
+	gma_sgx_aux_fini(dev);
 	gma_irq_uninstall(dev);
 
 	if (dev_priv->ops->chip_teardown)
@@ -337,29 +342,34 @@ static int psb_driver_load(struct drm_device *dev, unsigned long flags)
 
 	ret = -ENOMEM;
 
-	dev_priv->mmu = psb_mmu_driver_init(dev, 1, 0, NULL);
-	if (!dev_priv->mmu)
-		goto out_err;
+	/* No SGX MMU where the SGX has a driver of its own. */
+	if (!gma_sgx_external(dev_priv)) {
+		dev_priv->mmu = psb_mmu_driver_init(dev, 1, 0, NULL);
+		if (!dev_priv->mmu)
+			goto out_err;
 
-	dev_priv->pf_pd = psb_mmu_alloc_pd(dev_priv->mmu, 1, 0);
-	if (!dev_priv->pf_pd)
-		goto out_err;
+		dev_priv->pf_pd = psb_mmu_alloc_pd(dev_priv->mmu, 1, 0);
+		if (!dev_priv->pf_pd)
+			goto out_err;
+	}
 
 	ret = psb_do_init(dev);
 	if (ret)
 		return ret;
 
-	/* Add stolen memory to SGX MMU */
-	ret = psb_mmu_insert_pfn_sequence(psb_mmu_get_default_pd(dev_priv->mmu),
-					  dev_priv->stolen_base >> PAGE_SHIFT,
-					  pg->gatt_start,
-					  pg->stolen_size >> PAGE_SHIFT, 0);
+	if (!gma_sgx_external(dev_priv)) {
+		/* Add stolen memory to SGX MMU */
+		ret = psb_mmu_insert_pfn_sequence(psb_mmu_get_default_pd(dev_priv->mmu),
+						  dev_priv->stolen_base >> PAGE_SHIFT,
+						  pg->gatt_start,
+						  pg->stolen_size >> PAGE_SHIFT, 0);
 
-	psb_mmu_set_pd_context(psb_mmu_get_default_pd(dev_priv->mmu), 0);
-	psb_mmu_set_pd_context(dev_priv->pf_pd, 1);
+		psb_mmu_set_pd_context(psb_mmu_get_default_pd(dev_priv->mmu), 0);
+		psb_mmu_set_pd_context(dev_priv->pf_pd, 1);
 
-	PSB_WSGX32(0x20000000, PSB_CR_PDS_EXEC_BASE);
-	PSB_WSGX32(0x30000000, PSB_CR_BIF_3D_REQ_BASE);
+		PSB_WSGX32(0x20000000, PSB_CR_PDS_EXEC_BASE);
+		PSB_WSGX32(0x30000000, PSB_CR_BIF_3D_REQ_BASE);
+	}
 
 	acpi_video_register();
 
@@ -383,6 +393,8 @@ static int psb_driver_load(struct drm_device *dev, unsigned long flags)
 	spin_unlock_irqrestore(&dev_priv->irqmask_lock, irqflags);
 
 	gma_irq_install(dev);
+	if (dev_priv->ops->sgx)
+		gma_sgx_aux_init(dev);
 
 	dev->max_vblank_count = 0xffffff; /* only 24 bits of frame count */
 
diff --git a/drivers/gpu/drm/gma500/psb_drv.h b/drivers/gpu/drm/gma500/psb_drv.h
index db197b865..dbc5827aa 100644
--- a/drivers/gpu/drm/gma500/psb_drv.h
+++ b/drivers/gpu/drm/gma500/psb_drv.h
@@ -8,10 +8,12 @@
 #ifndef _PSB_DRV_H_
 #define _PSB_DRV_H_
 
+#include <linux/interrupt.h>
 #include <linux/kref.h>
 #include <linux/mm_types.h>
 
 #include <drm/drm_device.h>
+#include <drm/gma500_sgx.h>
 
 #include "gtt.h"
 #include "intel_bios.h"
@@ -407,6 +409,11 @@ struct drm_psb_private {
 
 	/* Register base */
 	uint8_t __iomem *sgx_reg;
+
+	/* The SGX's own driver, if one is bound (sgx_aux.c) */
+	struct gma500_sgx_device *sgx_adev;
+	gma500_sgx_irq_handler_t sgx_handler;
+	void *sgx_data;
 	uint8_t __iomem *vdc_reg;
 	uint8_t __iomem *aux_reg; /* Auxillary vdc pipe regs */
 	uint16_t lpc_gpio_base;
@@ -541,12 +548,18 @@ static inline struct drm_psb_private *to_drm_psb_private(struct drm_device *dev)
 	return container_of(dev, struct drm_psb_private, dev);
 }
 
+/* An SGX that gma500 publishes for a driver of its own (sgx_aux.c) */
+struct gma_sgx_info {
+	unsigned int core;	/* SGX core number: 545 for an SGX545 */
+};
+
 /* Operations for each board type */
 struct psb_ops {
 	const char *name;
 	int pipes;		/* Number of output pipes */
 	int crtcs;		/* Number of CRTCs */
 	int sgx_offset;		/* Base offset of SGX device */
+	const struct gma_sgx_info *sgx;	/* SGX for a driver of its own, or NULL */
 	int hdmi_mask;		/* Mask of HDMI CRTCs */
 	int lvds_mask;		/* Mask of LVDS CRTCs */
 	int sdvo_mask;		/* Mask of SDVO CRTCs */
@@ -719,10 +732,23 @@ static inline void REGISTER_WRITE8(struct drm_device *dev,
 #define PSB_WVDC32(_val, _offs)		iowrite32(_val, dev_priv->vdc_reg + (_offs))
 #define PSB_RVDC32(_offs)		ioread32(dev_priv->vdc_reg + (_offs))
 
+/*
+ * The chip's SGX has a driver of its own (psb_ops.sgx): gma500 never sets it
+ * up, maps nothing in its MMU and writes none of its registers.
+ */
+static inline bool gma_sgx_external(struct drm_psb_private *dev_priv)
+{
+	return dev_priv->ops->sgx;
+}
+
 #define PSB_RSGX32(_offs)		ioread32(dev_priv->sgx_reg + (_offs))
 #define PSB_WSGX32(_val, _offs)		iowrite32(_val, dev_priv->sgx_reg + (_offs))
 
 #define PSB_WMSVDX32(_val, _offs)	iowrite32(_val, dev_priv->msvdx_reg + (_offs))
 #define PSB_RMSVDX32(_offs)		ioread32(dev_priv->msvdx_reg + (_offs))
 
+/* sgx_aux.c */
+void gma_sgx_aux_init(struct drm_device *dev);
+void gma_sgx_aux_fini(struct drm_device *dev);
+
 #endif
diff --git a/drivers/gpu/drm/gma500/psb_irq.c b/drivers/gpu/drm/gma500/psb_irq.c
index c224c7ff3..426efc126 100644
--- a/drivers/gpu/drm/gma500/psb_irq.c
+++ b/drivers/gpu/drm/gma500/psb_irq.c
@@ -224,10 +224,19 @@ static irqreturn_t gma_irq_handler(int irq, void *arg)
 	}
 
 	if (sgx_int) {
-		sgx_stat_1 = PSB_RSGX32(PSB_CR_EVENT_STATUS);
-		sgx_stat_2 = PSB_RSGX32(PSB_CR_EVENT_STATUS2);
-		gma_sgx_interrupt(dev, sgx_stat_1, sgx_stat_2);
-		handled = 1;
+		if (gma_sgx_external(dev_priv)) {
+			gma500_sgx_irq_handler_t sgx_handler;
+
+			/* Pairs with the release in gma_sgx_set_irq_handler(). */
+			sgx_handler = smp_load_acquire(&dev_priv->sgx_handler);
+			if (sgx_handler && sgx_handler(dev_priv->sgx_data) != IRQ_NONE)
+				handled = 1;
+		} else {
+			sgx_stat_1 = PSB_RSGX32(PSB_CR_EVENT_STATUS);
+			sgx_stat_2 = PSB_RSGX32(PSB_CR_EVENT_STATUS2);
+			gma_sgx_interrupt(dev, sgx_stat_1, sgx_stat_2);
+			handled = 1;
+		}
 	}
 
 	/* Note: this bit has other meanings on some devices, so we will
@@ -257,8 +266,11 @@ void gma_irq_preinstall(struct drm_device *dev)
 	PSB_WVDC32(0xFFFFFFFF, PSB_HWSTAM);
 	PSB_WVDC32(0x00000000, PSB_INT_MASK_R);
 	PSB_WVDC32(0x00000000, PSB_INT_ENABLE_R);
-	PSB_WSGX32(0x00000000, PSB_CR_EVENT_HOST_ENABLE);
-	PSB_RSGX32(PSB_CR_EVENT_HOST_ENABLE);
+	/* An SGX driver of its own enables what it needs; until one binds, mask it. */
+	if (!gma_sgx_external(dev_priv) || !READ_ONCE(dev_priv->sgx_handler)) {
+		PSB_WSGX32(0x00000000, PSB_CR_EVENT_HOST_ENABLE);
+		PSB_RSGX32(PSB_CR_EVENT_HOST_ENABLE);
+	}
 
 	if (dev->vblank[0].enabled)
 		dev_priv->vdc_irq_mask |= _PSB_VSYNC_PIPEA_FLAG;
@@ -284,9 +296,11 @@ void gma_irq_postinstall(struct drm_device *dev)
 	spin_lock_irqsave(&dev_priv->irqmask_lock, irqflags);
 
 	/* Enable 2D and MMU fault interrupts */
-	PSB_WSGX32(_PSB_CE2_BIF_REQUESTER_FAULT, PSB_CR_EVENT_HOST_ENABLE2);
-	PSB_WSGX32(_PSB_CE_TWOD_COMPLETE, PSB_CR_EVENT_HOST_ENABLE);
-	PSB_RSGX32(PSB_CR_EVENT_HOST_ENABLE); /* Post */
+	if (!gma_sgx_external(dev_priv)) {
+		PSB_WSGX32(_PSB_CE2_BIF_REQUESTER_FAULT, PSB_CR_EVENT_HOST_ENABLE2);
+		PSB_WSGX32(_PSB_CE_TWOD_COMPLETE, PSB_CR_EVENT_HOST_ENABLE);
+		PSB_RSGX32(PSB_CR_EVENT_HOST_ENABLE); /* Post */
+	}
 
 	/* This register is safe even if display island is off */
 	PSB_WVDC32(dev_priv->vdc_irq_mask, PSB_INT_ENABLE_R);
diff --git a/drivers/gpu/drm/gma500/sgx_aux.c b/drivers/gpu/drm/gma500/sgx_aux.c
new file mode 100644
index 000000000..671cb6c33
--- /dev/null
+++ b/drivers/gpu/drm/gma500/sgx_aux.c
@@ -0,0 +1,102 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * The SGX of a GMA500-family GPU, published as an auxiliary device for a
+ * driver of its own on the chips whose psb_ops name it (Cedarview). gma500
+ * keeps the PCI function and the interrupt line, hands the SGX's interrupt
+ * over, and never sets the SGX up or touches its registers on those chips.
+ *
+ * Copyright © 2026 Joseph Telaak
+ */
+
+#include <linux/pci.h>
+#include <linux/slab.h>
+
+#include <drm/drm_print.h>
+#include <drm/gma500_sgx.h>
+
+#include "psb_drv.h"
+
+struct gma_sgx_aux {
+	struct gma500_sgx_device sgx;
+	struct drm_psb_private *dev_priv;
+};
+
+static int gma_sgx_set_irq_handler(struct gma500_sgx_device *sgx,
+				   gma500_sgx_irq_handler_t handler, void *data)
+{
+	struct gma_sgx_aux *aux = container_of(sgx, struct gma_sgx_aux, sgx);
+	struct drm_psb_private *dev_priv = aux->dev_priv;
+	struct pci_dev *pdev = to_pci_dev(dev_priv->dev.dev);
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev_priv->irqmask_lock, flags);
+	if (handler && dev_priv->sgx_handler) {
+		spin_unlock_irqrestore(&dev_priv->irqmask_lock, flags);
+		return -EBUSY;
+	}
+	dev_priv->sgx_data = data;
+	/* The data reaches the interrupt handler before the handler does. */
+	smp_store_release(&dev_priv->sgx_handler, handler);
+	spin_unlock_irqrestore(&dev_priv->irqmask_lock, flags);
+
+	/* No handler call still running on another CPU once this returns. */
+	if (!handler && dev_priv->irq_enabled)
+		synchronize_irq(pdev->irq);
+	return 0;
+}
+
+static void gma_sgx_aux_release(struct device *dev)
+{
+	struct auxiliary_device *adev = to_auxiliary_dev(dev);
+
+	kfree(container_of(to_gma500_sgx_device(adev), struct gma_sgx_aux, sgx));
+}
+
+void gma_sgx_aux_init(struct drm_device *dev)
+{
+	struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
+	struct gma_sgx_aux *aux;
+	int ret;
+
+	aux = kzalloc_obj(*aux);
+	if (!aux) {
+		drm_warn(dev, "SGX auxiliary device not published (-ENOMEM)\n");
+		return;
+	}
+	aux->dev_priv = dev_priv;
+	aux->sgx.regs = dev_priv->sgx_reg;
+	aux->sgx.regs_len = PSB_SGX_SIZE;
+	aux->sgx.core = dev_priv->ops->sgx->core;
+	aux->sgx.set_irq_handler = gma_sgx_set_irq_handler;
+	aux->sgx.adev.name = "sgx";
+	aux->sgx.adev.id = pci_dev_id(to_pci_dev(dev->dev));
+	aux->sgx.adev.dev.parent = dev->dev;
+	aux->sgx.adev.dev.release = gma_sgx_aux_release;
+
+	ret = auxiliary_device_init(&aux->sgx.adev);
+	if (ret) {
+		kfree(aux);
+		drm_warn(dev, "SGX auxiliary device not published (%d)\n", ret);
+		return;
+	}
+	ret = auxiliary_device_add(&aux->sgx.adev);
+	if (ret) {
+		auxiliary_device_uninit(&aux->sgx.adev);
+		drm_warn(dev, "SGX auxiliary device not published (%d)\n", ret);
+		return;
+	}
+	dev_priv->sgx_adev = &aux->sgx;
+}
+
+void gma_sgx_aux_fini(struct drm_device *dev)
+{
+	struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
+
+	if (!dev_priv->sgx_adev)
+		return;
+	/* Unbinding the SGX driver hands the interrupt back; make sure of it. */
+	auxiliary_device_delete(&dev_priv->sgx_adev->adev);
+	gma_sgx_set_irq_handler(dev_priv->sgx_adev, NULL, NULL);
+	auxiliary_device_uninit(&dev_priv->sgx_adev->adev);
+	dev_priv->sgx_adev = NULL;
+}
diff --git a/include/drm/gma500_sgx.h b/include/drm/gma500_sgx.h
new file mode 100644
index 000000000..00a034f1a
--- /dev/null
+++ b/include/drm/gma500_sgx.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: MIT */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef __DRM_GMA500_SGX_H__
+#define __DRM_GMA500_SGX_H__
+
+#include <linux/auxiliary_bus.h>
+#include <linux/interrupt.h>
+#include <linux/types.h>
+
+/* The SGX's interrupt handler: hard interrupt context. */
+typedef irqreturn_t (*gma500_sgx_irq_handler_t)(void *data);
+
+/**
+ * struct gma500_sgx_device - The PowerVR SGX inside a GMA500-family GPU
+ *
+ * gma500 owns the PCI function, its register BAR and its interrupt. On parts
+ * whose SGX has a driver of its own (&psb_ops.sgx), it publishes this auxiliary
+ * device, named "gma500_gfx.sgx", whose parent is the GPU's PCI device. The SGX
+ * driver uses the parent for DMA and runtime PM: holding the auxiliary device
+ * awake keeps gma500, and so the graphics power island, awake.
+ */
+struct gma500_sgx_device {
+	/** @adev: The auxiliary device. */
+	struct auxiliary_device adev;
+
+	/** @regs: The SGX's register window, mapped by gma500. */
+	void __iomem *regs;
+
+	/** @regs_len: Length of @regs in bytes. */
+	resource_size_t regs_len;
+
+	/** @core: Which SGX core this is, by number: 545 for an SGX545. */
+	unsigned int core;
+
+	/**
+	 * @set_irq_handler: Route the SGX's interrupts to @handler, which runs in
+	 * hard interrupt context with @data; with a NULL @handler they are
+	 * ignored. gma500 never touches an SGX it publishes, handler or not.
+	 * Returns -EBUSY if another handler is set.
+	 */
+	int (*set_irq_handler)(struct gma500_sgx_device *sgx,
+			       gma500_sgx_irq_handler_t handler, void *data);
+};
+
+static inline struct gma500_sgx_device *
+to_gma500_sgx_device(struct auxiliary_device *adev)
+{
+	return container_of(adev, struct gma500_sgx_device, adev);
+}
+
+#endif /* __DRM_GMA500_SGX_H__ */

-- 
2.50.1

[PATCH RFC 03/13] drm/gma500: Export GEM objects as dma-bufs

[Joseph Telaak]

A render GPU next to gma500 draws into gma500's scanout buffers: on
Cedarview the SGX545, driven by powervr_sgx through gma500's auxiliary
device, renders while gma500 scans out. Mesa's kmsro allocates the
buffers on the display device and imports them into the render device,
which needs gma500 to export them.

Add the pin, unpin, get_sg_table, vmap and vunmap hooks the PRIME
helpers need. An export pins the object's pages, write-combined as for a
framebuffer, but doesn't insert them into the GTT: the importer maps the
pages itself, and powervr_sgx takes a write-combined kernel mapping of
an import for its cache maintenance. (A CPU mmap through
dma_buf_mmap() still goes through psb_gem_fault(), which pins the object
into the GTT.) A use count shares the pages between GTT pins, exports
and kernel mappings. Objects in stolen memory have no struct pages and
can't be exported.

The PRIME helpers call pin and unpin with the object's reservation lock
held, and psb_gem_pin() and psb_gem_unpin() take that lock themselves.
Split each into a variant that expects the lock held.

The importer's CPU mapping, dma_buf_mmap(), goes through the object's
fake mmap offset, which a dumb buffer only gets from MAP_DUMB, and Mesa's
kmsro never calls that for the buffers it exports. Create the offset at
export.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/gma500/gem.c | 182 ++++++++++++++++++++++++++++++++++++-------
 drivers/gpu/drm/gma500/gem.h |   1 +
 2 files changed, 153 insertions(+), 30 deletions(-)

diff --git a/drivers/gpu/drm/gma500/gem.c b/drivers/gpu/drm/gma500/gem.c
index 4a4ecfb7b..9eff7a739 100644
--- a/drivers/gpu/drm/gma500/gem.c
+++ b/drivers/gpu/drm/gma500/gem.c
@@ -11,11 +11,14 @@
  *		accelerated operations on a GEM object)
  */
 
+#include <linux/iosys-map.h>
 #include <linux/pagemap.h>
+#include <linux/vmalloc.h>
 
 #include <asm/set_memory.h>
 
 #include <drm/drm.h>
+#include <drm/drm_prime.h>
 #include <drm/drm_print.h>
 #include <drm/drm_vma_manager.h>
 
@@ -26,71 +29,104 @@
  * PSB GEM object
  */
 
-int psb_gem_pin(struct psb_gem_object *pobj)
+/*
+ * The object's pages, write-combined, for a GART pin or a PRIME export: an
+ * exported object needs them, not a slot in the small GTT aperture.
+ */
+static int psb_gem_get_pages_locked(struct psb_gem_object *pobj)
+{
+	struct drm_gem_object *obj = &pobj->base;
+	struct page **pages;
+
+	dma_resv_assert_held(obj->resv);
+
+	if (pobj->pages_use++)
+		return 0;
+	pages = drm_gem_get_pages(obj);
+	if (IS_ERR(pages)) {
+		pobj->pages_use--;
+		return PTR_ERR(pages);
+	}
+	set_pages_array_wc(pages, obj->size / PAGE_SIZE);
+	pobj->pages = pages;
+	return 0;
+}
+
+static void psb_gem_put_pages_locked(struct psb_gem_object *pobj)
+{
+	struct drm_gem_object *obj = &pobj->base;
+
+	dma_resv_assert_held(obj->resv);
+
+	if (WARN_ON(!pobj->pages_use) || --pobj->pages_use)
+		return;
+	/* Reset caching flags */
+	set_pages_array_wb(pobj->pages, obj->size / PAGE_SIZE);
+	drm_gem_put_pages(obj, pobj->pages, true, false);
+	pobj->pages = NULL;
+}
+
+static int psb_gem_pin_locked(struct psb_gem_object *pobj)
 {
 	struct drm_gem_object *obj = &pobj->base;
 	struct drm_device *dev = obj->dev;
 	struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
 	u32 gpu_base = dev_priv->gtt.gatt_start;
-	struct page **pages;
 	unsigned int npages;
 	int ret;
 
-	ret = dma_resv_lock(obj->resv, NULL);
-	if (drm_WARN_ONCE(dev, ret, "dma_resv_lock() failed, ret=%d\n", ret))
-		return ret;
+	dma_resv_assert_held(obj->resv);
 
 	if (pobj->in_gart || pobj->stolen)
 		goto out; /* already mapped */
 
-	pages = drm_gem_get_pages(obj);
-	if (IS_ERR(pages)) {
-		ret = PTR_ERR(pages);
-		goto err_dma_resv_unlock;
-	}
+	ret = psb_gem_get_pages_locked(pobj);
+	if (ret)
+		return ret;
 
 	npages = obj->size / PAGE_SIZE;
 
-	set_pages_array_wc(pages, npages);
-
-	psb_gtt_insert_pages(dev_priv, &pobj->resource, pages);
+	psb_gtt_insert_pages(dev_priv, &pobj->resource, pobj->pages);
 	if (dev_priv->mmu)
-		psb_mmu_insert_pages(psb_mmu_get_default_pd(dev_priv->mmu), pages,
+		psb_mmu_insert_pages(psb_mmu_get_default_pd(dev_priv->mmu), pobj->pages,
 				     (gpu_base + pobj->offset), npages, 0, 0,
 				     PSB_MMU_CACHED_MEMORY);
 
-	pobj->pages = pages;
-
 out:
 	++pobj->in_gart;
-	dma_resv_unlock(obj->resv);
-
 	return 0;
+}
+
+int psb_gem_pin(struct psb_gem_object *pobj)
+{
+	struct drm_gem_object *obj = &pobj->base;
+	int ret;
 
-err_dma_resv_unlock:
+	ret = dma_resv_lock(obj->resv, NULL);
+	if (drm_WARN_ONCE(obj->dev, ret, "dma_resv_lock() failed, ret=%d\n", ret))
+		return ret;
+	ret = psb_gem_pin_locked(pobj);
 	dma_resv_unlock(obj->resv);
+
 	return ret;
 }
 
-void psb_gem_unpin(struct psb_gem_object *pobj)
+static void psb_gem_unpin_locked(struct psb_gem_object *pobj)
 {
 	struct drm_gem_object *obj = &pobj->base;
 	struct drm_device *dev = obj->dev;
 	struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
 	u32 gpu_base = dev_priv->gtt.gatt_start;
 	unsigned long npages;
-	int ret;
 
-	ret = dma_resv_lock(obj->resv, NULL);
-	if (drm_WARN_ONCE(dev, ret, "dma_resv_lock() failed, ret=%d\n", ret))
-		return;
+	dma_resv_assert_held(obj->resv);
 
 	WARN_ON(!pobj->in_gart);
 
 	--pobj->in_gart;
 
 	if (pobj->in_gart || pobj->stolen)
-		goto out;
+		return;
 
 	npages = obj->size / PAGE_SIZE;
 
@@ -99,13 +135,18 @@ void psb_gem_unpin(struct psb_gem_object *pobj)
 				     (gpu_base + pobj->offset), npages, 0, 0);
 	psb_gtt_remove_pages(dev_priv, &pobj->resource);
 
-	/* Reset caching flags */
-	set_pages_array_wb(pobj->pages, npages);
+	psb_gem_put_pages_locked(pobj);
+}
 
-	drm_gem_put_pages(obj, pobj->pages, true, false);
-	pobj->pages = NULL;
+void psb_gem_unpin(struct psb_gem_object *pobj)
+{
+	struct drm_gem_object *obj = &pobj->base;
+	int ret;
 
-out:
+	ret = dma_resv_lock(obj->resv, NULL);
+	if (drm_WARN_ONCE(obj->dev, ret, "dma_resv_lock() failed, ret=%d\n", ret))
+		return;
+	psb_gem_unpin_locked(pobj);
 	dma_resv_unlock(obj->resv);
 }
 
@@ -133,8 +174,89 @@ static const struct vm_operations_struct psb_gem_vm_ops = {
 	.close = drm_gem_vm_close,
 };
 
+/*
+ * PRIME export, for a render GPU to draw into scanout buffers. Only objects
+ * with pages can be shared: stolen memory has none. An export pins the pages,
+ * write-combined, and doesn't insert them into the GTT. The PRIME helpers
+ * call pin and unpin with the reservation lock held.
+ */
+static int psb_gem_prime_pin(struct drm_gem_object *obj)
+{
+	struct psb_gem_object *pobj = to_psb_gem_object(obj);
+
+	if (pobj->stolen)
+		return -EINVAL;
+	return psb_gem_get_pages_locked(pobj);
+}
+
+static void psb_gem_prime_unpin(struct drm_gem_object *obj)
+{
+	psb_gem_put_pages_locked(to_psb_gem_object(obj));
+}
+
+static struct sg_table *psb_gem_prime_get_sg_table(struct drm_gem_object *obj)
+{
+	struct psb_gem_object *pobj = to_psb_gem_object(obj);
+
+	if (!pobj->pages)
+		return ERR_PTR(-EINVAL);
+	return drm_prime_pages_to_sg(obj->dev, pobj->pages, obj->size >> PAGE_SHIFT);
+}
+
+/*
+ * A kernel mapping for an importer, write-combined like the pages. It holds
+ * the pages as a PRIME pin does. The PRIME helpers call it with the
+ * reservation lock held.
+ */
+static int psb_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map)
+{
+	struct psb_gem_object *pobj = to_psb_gem_object(obj);
+	void *vaddr;
+	int ret;
+
+	if (pobj->stolen)
+		return -EINVAL;
+	ret = psb_gem_get_pages_locked(pobj);
+	if (ret)
+		return ret;
+	vaddr = vmap(pobj->pages, obj->size >> PAGE_SHIFT, VM_MAP,
+		     pgprot_writecombine(PAGE_KERNEL));
+	if (!vaddr) {
+		psb_gem_put_pages_locked(pobj);
+		return -ENOMEM;
+	}
+	iosys_map_set_vaddr(map, vaddr);
+	return 0;
+}
+
+static void psb_gem_prime_vunmap(struct drm_gem_object *obj, struct iosys_map *map)
+{
+	vunmap(map->vaddr);
+	psb_gem_put_pages_locked(to_psb_gem_object(obj));
+}
+
+/*
+ * dma_buf_mmap() maps through the object's fake offset (drm_gem_prime_mmap()),
+ * which a dumb buffer only gets from MAP_DUMB: make sure it has one.
+ */
+static struct dma_buf *psb_gem_prime_export(struct drm_gem_object *obj, int flags)
+{
+	int ret;
+
+	ret = drm_gem_create_mmap_offset(obj);
+	if (ret)
+		return ERR_PTR(ret);
+	return drm_gem_prime_export(obj, flags);
+}
+
 static const struct drm_gem_object_funcs psb_gem_object_funcs = {
 	.free = psb_gem_free_object,
+	.export = psb_gem_prime_export,
+	.pin = psb_gem_prime_pin,
+	.unpin = psb_gem_prime_unpin,
+	.get_sg_table = psb_gem_prime_get_sg_table,
+	.vmap = psb_gem_prime_vmap,
+	.vunmap = psb_gem_prime_vunmap,
 	.vm_ops = &psb_gem_vm_ops,
 };
 
diff --git a/drivers/gpu/drm/gma500/gem.h b/drivers/gpu/drm/gma500/gem.h
index 27df6ff4e..539f8f20a 100644
--- a/drivers/gpu/drm/gma500/gem.h
+++ b/drivers/gpu/drm/gma500/gem.h
@@ -27,6 +27,7 @@ struct psb_gem_object {
 	bool stolen;			/* Backed from stolen RAM */
 	bool mmapping;			/* Is mmappable */
 	struct page **pages;		/* Backing pages if present */
+	unsigned int pages_use;		/* GART pins and PRIME pins of @pages */
 };
 
 static inline struct psb_gem_object *to_psb_gem_object(struct drm_gem_object *obj)

-- 
2.50.1

[PATCH RFC 04/13] drm/powervr_sgx: Add the uAPI header and its documentation

[Joseph Telaak]

The uAPI of powervr_sgx, the render-only driver for PowerVR SGX5xx
(Series5) GPUs that the next patch adds. It follows the rules of the
Rogue driver's uAPI: type-aligned members, explicit padding, zeroed
reserved fields, one layout on i386 and x86_64. The DRM version
starts at 1.0 and the ioctl numbers are dense from 0.

- DEV_QUERY: the GPU's identity, limits, quirks and heaps.
- CREATE_BO, GET_BO_MMAP_OFFSET: shmem objects below 4 GiB. A code
  object has no CPU mapping; UPLOAD_CODE fills it, and the kernel checks
  every instruction first.
- CREATE_VM_CONTEXT, VM_BIND: one GPU page directory per VM, with
  fixed heaps; VM_BIND maps and unmaps synchronously and takes no
  fences.
- CREATE_CONTEXT, GET_CONTEXT_STATE, CREATE_RT_DATASET: render
  contexts, render target data, and the guilty/innocent state after a
  reset.
- SUBMIT_JOBS: render jobs with syncobj waits and signals. The
  header also has a transfer context and job (a 3D-only blit) and a
  NULL job, which the driver's test tools use. Mesa uses neither, and
  a client's first transfer can fault after another client's transfers
  have run. Both go before a v1 unless a reviewer wants them.
  A job's commands are plain structs, with an extension chain for the
  fields only one core has; the kernel copies them, checks every field
  and builds the firmware's command from them. Userspace
  never writes a firmware structure. Sync is explicit, and a job lists
  the byte ranges of imported buffers it reads or writes, for the
  kernel's cache maintenance.

Every ioctl is render-node only. Documentation/gpu/powervr_sgx has the
overview and the uAPI; MAINTAINERS gets an entry.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 Documentation/gpu/drivers.rst           |    1 +
 Documentation/gpu/powervr_sgx/index.rst |  201 ++++++
 Documentation/gpu/powervr_sgx/uapi.rst  |  133 ++++
 MAINTAINERS                             |   10 +
 include/uapi/drm/powervr_sgx_drm.h      | 1175 +++++++++++++++++++++++++++++++
 5 files changed, 1520 insertions(+)

diff --git a/Documentation/gpu/drivers.rst b/Documentation/gpu/drivers.rst
index 20d2c454a..d1a2a3da5 100644
--- a/Documentation/gpu/drivers.rst
+++ b/Documentation/gpu/drivers.rst
@@ -8,6 +8,7 @@ GPU Driver Documentation
    amdgpu/index
    i915
    imagination/index
+   powervr_sgx/index
    intel-display/index
    mcde
    meson
diff --git a/Documentation/gpu/powervr_sgx/index.rst b/Documentation/gpu/powervr_sgx/index.rst
new file mode 100644
index 000000000..ce21abb9d
--- /dev/null
+++ b/Documentation/gpu/powervr_sgx/index.rst
@@ -0,0 +1,201 @@
+.. SPDX-License-Identifier: GPL-2.0-only OR MIT
+
+===============================================
+drm/powervr_sgx PowerVR SGX545 GPU Driver
+===============================================
+
+``powervr_sgx`` is a render-only DRM driver for the PowerVR SGX545. It binds
+the GPU where it is a PCI function of its own, as in the Intel CE5300
+(``8086:089b``, core revision 1.0.14), and on Cedarview (GMA3600/3650, core
+revision 1.0.13), where the SGX shares a PCI function with the display engine,
+as the auxiliary device ``gma500_gfx.sgx`` that gma500 publishes. Scanout goes
+through a separate KMS driver, and buffers move between the two as dma-bufs.
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents
+
+   uapi
+
+Firmware
+========
+
+The SGX545 runs a microkernel on its USSE, loaded with request_firmware() from
+``powervr_sgx/``. The microkernel takes commands from a kernel-owned command
+buffer, kicks the tile accelerator (TA) and the 3D core, and raises an
+interrupt when a job is done. The driver builds the microkernel's data
+structures: the host control block, the command buffer, render and transfer
+contexts, render target datasets and parameter buffers. Userspace never writes
+them.
+
+The supported cores are in a table (``pvrsgx_core.c``). Each core design has
+an entry with its ID, the registers whose offsets or fields differ between
+SGX5xx cores, its features, instruction set, directory lists, address width
+and init script. Each part of it has an entry with its revision, clock and
+firmware file. The bus tells the driver which core to expect, since even the
+core ID register moves between cores: the PCI ID for the CE5300, the gma500
+auxiliary device for Cedarview. The driver refuses a core that isn't in the
+table before writing any register.
+For the CE5300 the file is ``powervr_sgx/sgx545_ukernel.bin``. It starts
+with a header giving the host interface version, feature bits, the image's CRC,
+its entry points and the immediates the driver fills with buffer addresses.
+The driver only loads an image that declares secure mode, in which no client
+page directory maps the firmware's memory. After the start it asks the
+microkernel for its version and features.
+
+Memory
+======
+
+Buffer objects are shmem GEM objects with pages below 4 GiB, since the GPU's
+physical addresses are 32 bits wide. CPU mappings are write-combined. Every
+open file creates VM contexts. Each one has its own two-level page directory
+and is managed with drm_gpuvm. The client heaps sit at fixed device addresses,
+reported by ``DEV_QUERY``. Kernel-owned objects live in ranges a client can't
+map.
+
+USSE programs run only from code objects. A code object has no CPU mapping and
+can't be exported, and the GPU sees it read-only. Its contents arrive through
+``UPLOAD_CODE``, which validates them (see `Shader validation`_).
+
+dma-bufs are exported and imported with the shmem helpers. An imported object
+maps in any heap a data object maps in. A CPU mapping of an exported dma-buf
+needs ``DRM_POWERVR_SGX_BO_ALLOW_CPU_ACCESS``, as a mapping through the DRM
+file does.
+
+Shader validation
+=================
+
+The GPU's page tables keep a client's memory accesses to its own VM, but a USSE
+program can also read the core's own registers (``ldr``), which the page
+tables can't stop. On the SGX545, a client's fragment task read the core ID,
+a live timer and the page directory base. A client's ``str`` to the page
+directory base or to code bases 14 and 15 was dropped. Stores to other
+registers from a client task are untested, and the microkernel itself writes
+registers with ``str``, so stores are refused as well. So the kernel decides
+which USSE code may run for a client.
+
+**Where client code runs from.** A USSE task starts at one of the 16 USE code
+bases plus an offset, both taken from a ``DOUTU`` word of a PDS program. PDS
+programs and their data are client memory and are not checked, so a client can
+name any base and any offset. The offset, like the absolute targets inside
+USSE code, reaches at most 2 MiB past the base. The kernel programs bases 0-9
+onto the pixel shader heap, 10-13 onto the vertex shader heap, 14 the same as
+0, and 15 onto the microkernel's code, which no client page directory maps.
+From bases 0-14, everything within reach is a USSE heap or an address range no
+client can map. The USSE heaps take only code objects. So a client task runs
+code ``UPLOAD_CODE`` checked, or faults.
+
+**What UPLOAD_CODE checks.** Every 8-byte word of an upload, whether or not
+the program's branches reach it, against a table generated from the isaspec
+description of the instruction set that Mesa's compiler uses
+(``pvrsgx_usse_table.h``):
+
+- The word must be exactly one known form. Fields the compilers leave at one
+  value are pinned to it.
+- Refused outright: register-file access (``ldr``, ``str``); everything that
+  sets or uses the link address (``setl``, ``savl``, ``lapc``, a branch with
+  ``link``), since together they make a computed jump; the absolute branch
+  ``ba``; ``phas`` with a register operand; the mutexes ``lock`` and
+  ``release``, which can stall other tasks; the operand-stepping controls
+  whose effect on operand numbers is not known (``smbo``, ``smr``, ``smoa``,
+  ``imo`` and the register forms of all four with ``smlsi``); dual issue
+  (``fdual``), whose second operation's operands aren't checked; and forms
+  this core doesn't have or whose meaning is not known.
+- ``br``: the target must be inside the object.
+- ``phas.i``: a target other than 0 counts from the code base the task's PDS
+  program names. Measured: base 0 with offset 0x88380 runs code in window 1,
+  and its targets still count from base 0. The check takes the base a compiler
+  names, the 512 KiB window of the USSE heap the task's first instruction is
+  in, and the target must be inside the object where it is mapped. Such an
+  object must lie in one window, so that base is the same for every entry in
+  it, be mapped when its code is uploaded, at one place in its window only,
+  and can't be mapped at another place afterwards. A PDS program that names
+  another base only moves the target within the reach described above.
+- ``setfc``: only ``setfc #0, #1``, the format control a program starts
+  with, so the check knows the format control of every instruction without
+  following the program.
+- Each operand: no writes to constants or globals; no reads of the globals
+  past ``g45``, which are not documented; internal registers ``i0``-``i3``
+  and index registers ``idx1``-``idx2`` only. Bit 6 of an operand number is
+  a format flag only under format select (the instruction's ``fmt`` bit) or,
+  for the color classes' ``r``, ``o``, ``pa``, ``sa`` and ``i`` operands, C10
+  format control; everywhere else it is part of the number.
+
+**What is not checked, and why.** Memory accesses (``ld``, ``st``, texture
+samples, the pixel back end's tile writes through ``emit``, PDS DMA) go
+through the job's page directory, which maps only the client's own objects
+and the parameter buffer, and none of the firmware's. The GPU can't modify a
+code object once it is checked. Code objects are mapped read-only, and the MMU
+faults USSE stores and pixel back end writes to such pages.
+
+**Known gaps.**
+
+- The parameter buffer is one per device and mapped read-write into every
+  client VM, so one client's shader can read or corrupt the scenes other
+  clients are binning.
+- The following are expected but not yet tested on hardware: that a client
+  task's instruction fetch from base 15 faults, and that the read-only PTE
+  bit also stops the depth/stencil store and the tile accelerator.
+- Indexed operands and repeated instructions can step an operand past the
+  128 registers its field names. Whether the hardware keeps such accesses
+  inside the task's own registers is not documented. The compilers need
+  both.
+
+Jobs
+====
+
+One drm_gpu_scheduler feeds the microkernel, and contexts are scheduler
+entities. Unlike firmware-scheduled GPUs, where each queue gets a scheduler of
+its own, the SGX545's microkernel takes commands from a single kernel command
+buffer and doesn't choose between contexts, so drm_sched does: one job on the
+GPU at a time (credit limit 1), in priority order. After a reset the queued
+jobs simply run; nothing is resubmitted. A render job is a scene of one or two TA kicks and the 3D render that
+follows them. A transfer job is a blit. Userspace describes either with one
+struct per command and a chain of core-specific extensions, which the kernel
+copies and checks field by field, from tables generated per core, before it
+writes the microkernel's command. Jobs wait for and signal binary and timeline syncobjs;
+sync is explicit only, and a shared buffer's reservation gets no fence from the
+driver (userspace moves fences through the dma-buf's sync-file ioctls). The GPU
+doesn't snoop the CPU caches, so a job lists the bytes of imports it reads or
+writes, and the driver flushes those around it.
+
+Recovery
+========
+
+A job that doesn't finish within the scheduler's timeout, or that takes a GPU
+page fault, resets the GPU. The SGX545 raises no interrupt for a page fault, so
+the driver checks the MMU's fault status while jobs are on the GPU. The
+driver halts the cores and takes a devcoredump (registers, the microkernel's
+control blocks, the command buffer and the hung job's structures). Then it
+fails the jobs that were on the GPU: the guilty one with -ETIME, or -EFAULT
+after a fault, and the others with -ECANCELED. It restarts the microkernel and
+marks the guilty context. Its later submissions fail, and other contexts carry
+on.
+
+Power
+=====
+
+With runtime PM, a job on the GPU holds a reference. The device autosuspends
+500 ms after the last one. Suspend powers the microkernel off and holds the
+cores in reset; resume restarts it. The SGX function has no PCI PM capability,
+so it stays in D0. While the core is stopped the driver reads none of its
+registers.
+
+Debugging
+=========
+
+debugfs, under ``/sys/kernel/debug/dri/<minor>/``, has:
+
+``status``
+    firmware state, command buffer offsets, the host control block, the
+    kernel objects waiting to be freed, the client unmaps waiting for the
+    TLB invalidate and, while the GPU is awake, the core's registers.
+
+``ctlblk``
+    the microkernel's TA/3D and host control blocks.
+
+Module parameters
+=================
+
+``job_timeout_ms``
+    how long a job may run before the GPU is reset (default 2000).
diff --git a/Documentation/gpu/powervr_sgx/uapi.rst b/Documentation/gpu/powervr_sgx/uapi.rst
new file mode 100644
index 000000000..8dd790897
--- /dev/null
+++ b/Documentation/gpu/powervr_sgx/uapi.rst
@@ -0,0 +1,133 @@
+.. SPDX-License-Identifier: GPL-2.0-only OR MIT
+
+====
+UAPI
+====
+The sources associated with this section can be found in ``powervr_sgx_drm.h``.
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX UAPI
+
+OBJECT ARRAYS
+=============
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_obj_array
+
+IOCTLS
+======
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX IOCTL interface
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: PVR_SGX_IOCTL
+
+DEV_QUERY
+---------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX IOCTL DEV_QUERY interface
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_dev_query
+                 drm_powervr_sgx_ioctl_dev_query_args
+                 drm_powervr_sgx_dev_query_gpu_info
+                 drm_powervr_sgx_dev_query_limits
+                 drm_powervr_sgx_dev_query_commands
+                 drm_powervr_sgx_dev_query_quirks
+                 drm_powervr_sgx_heap_id
+                 drm_powervr_sgx_heap
+                 drm_powervr_sgx_dev_query_heap_info
+
+CREATE_BO
+---------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX IOCTL CREATE_BO interface
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_ioctl_create_bo_args
+                 drm_powervr_sgx_ioctl_get_bo_mmap_offset_args
+
+UPLOAD_CODE
+-----------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX IOCTL UPLOAD_CODE interface
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_ioctl_upload_code_args
+
+VM CONTEXTS
+-----------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX IOCTL VM context interfaces
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_ioctl_create_vm_context_args
+                 drm_powervr_sgx_ioctl_destroy_vm_context_args
+
+VM_BIND
+-------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: VM_BIND operations
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: Flags for struct drm_powervr_sgx_ioctl_vm_bind_args
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_vm_bind_op
+                 drm_powervr_sgx_ioctl_vm_bind_args
+
+CONTEXTS
+--------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX IOCTL CREATE_CONTEXT interface
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_ctx_type
+                 drm_powervr_sgx_ctx_priority
+                 drm_powervr_sgx_ioctl_create_context_args
+                 drm_powervr_sgx_ioctl_destroy_context_args
+                 drm_powervr_sgx_context_state
+                 drm_powervr_sgx_ioctl_get_context_state_args
+
+RT DATASETS
+-----------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX IOCTL CREATE_RT_DATASET interface
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_ioctl_create_rt_dataset_args
+                 drm_powervr_sgx_ioctl_destroy_rt_dataset_args
+
+CLIENT COMMANDS
+---------------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX client commands
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_cmd_head
+                 drm_powervr_sgx_ext
+                 drm_powervr_sgx_cmd_render
+                 drm_powervr_sgx_cmd_ta_kick
+                 drm_powervr_sgx_cmd_transfer
+                 drm_powervr_sgx_ext_sgx545_render
+                 drm_powervr_sgx_ext_sgx545_ta_kick
+
+SUBMIT_JOBS
+-----------
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: PowerVR SGX IOCTL SUBMIT_JOBS interface
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: Flags for struct drm_powervr_sgx_sync_op
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: Flags for struct drm_powervr_sgx_cache_range
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :doc: Flags for &drm_powervr_sgx_job.flags
+
+.. kernel-doc:: include/uapi/drm/powervr_sgx_drm.h
+   :identifiers: drm_powervr_sgx_sync_op
+                 drm_powervr_sgx_cache_range
+                 drm_powervr_sgx_job_type
+                 drm_powervr_sgx_job
+                 drm_powervr_sgx_ioctl_submit_jobs_args
diff --git a/MAINTAINERS b/MAINTAINERS
index e1b0a4245..30ba6f6a4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8635,6 +8635,16 @@ S:	Maintained
 F:	Documentation/devicetree/bindings/display/mayqueen,pixpaper.yaml
 F:	drivers/gpu/drm/tiny/pixpaper.c
 
+DRM DRIVER FOR POWERVR SGX5XX GPUS
+M:	Joseph Telaak <joseph@telaak.dev>
+L:	dri-devel@lists.freedesktop.org
+S:	Maintained
+T:	git https://gitlab.freedesktop.org/drm/misc/kernel.git
+F:	Documentation/gpu/powervr_sgx/
+F:	drivers/gpu/drm/powervr_sgx/
+F:	include/drm/gma500_sgx.h
+F:	include/uapi/drm/powervr_sgx_drm.h
+
 DRM DRIVER FOR QEMU'S CIRRUS DEVICE
 M:	Dave Airlie <airlied@redhat.com>
 M:	Gerd Hoffmann <kraxel@redhat.com>
diff --git a/include/uapi/drm/powervr_sgx_drm.h b/include/uapi/drm/powervr_sgx_drm.h
new file mode 100644
index 000000000..123818095
--- /dev/null
+++ b/include/uapi/drm/powervr_sgx_drm.h
@@ -0,0 +1,1175 @@
+/* SPDX-License-Identifier: (GPL-2.0-only WITH Linux-syscall-note) OR MIT */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef POWERVR_SGX_DRM_H
+#define POWERVR_SGX_DRM_H
+
+#include "drm.h"
+
+#include <linux/const.h>
+#include <linux/types.h>
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+/**
+ * DOC: PowerVR SGX UAPI
+ *
+ * The DRM driver is ``powervr_sgx``. Identifiers use the same name in full,
+ * ``drm_powervr_sgx_`` and ``DRM_IOCTL_POWERVR_SGX_``, because ``pvr`` is the
+ * upstream Rogue driver's prefix.
+ *
+ * The argument structs follow the rules of the upstream PowerVR uAPI:
+ *
+ *  - All members are type-aligned, on i386 as on x86_64.
+ *  - Every struct is padded to 64-bit alignment.
+ *  - Padding is explicit, in ``_padding_x`` members, where x is the member's
+ *    offset in hexadecimal. Padding and reserved fields must be zero; the
+ *    kernel always checks them and returns -EINVAL otherwise.
+ *  - User pointers and device-virtual addresses are __u64. A device-virtual
+ *    address at or above 2^&drm_powervr_sgx_dev_query_limits.va_bits is
+ *    rejected.
+ *  - Structs reached through &struct drm_powervr_sgx_obj_array can grow at the
+ *    end. A short object is zero-extended. In an object longer than the kernel
+ *    knows, the unknown bytes must be zero.
+ *  - Client commands grow the same way, through &drm_powervr_sgx_cmd_head.size,
+ *    and carry core-specific fields in extensions: see "PowerVR SGX client
+ *    commands" below.
+ *
+ * The DRM driver's version is 1.0. Additions to the interface raise the minor
+ * version; nothing is removed or renumbered.
+ */
+
+/**
+ * struct drm_powervr_sgx_obj_array - Container used to pass arrays of objects
+ *
+ * The stride versions the element type. A stride smaller than the kernel's
+ * element size zero-extends each element; a larger one requires the unknown
+ * bytes to be zero. When the kernel writes elements back it writes the smaller
+ * of the two sizes.
+ */
+struct drm_powervr_sgx_obj_array {
+	/** @stride: Size of one element, in bytes. */
+	__u32 stride;
+
+	/** @count: Number of elements. */
+	__u32 count;
+
+	/** @array: User pointer to the elements. */
+	__u64 array;
+};
+
+/**
+ * DOC: PowerVR SGX IOCTL interface
+ *
+ * Every ioctl below is valid on the render node and is marked DRM_RENDER_ALLOW.
+ * The driver has no modeset or DRM-master ioctls.
+ *
+ * Common errors, in addition to the ones listed per ioctl:
+ *
+ *  - -EINVAL: a reserved field or flag is not zero, or an argument is invalid.
+ *  - -ENOENT: a handle does not name a live object of the required type.
+ *  - -EFAULT: a user pointer cannot be read or written.
+ *  - -ENOSPC: the file already has 64 contexts or 64 RT datasets, or the
+ *    device's kernel memory ranges are full.
+ *  - -EIO: the GPU could not be recovered by a reset (device lost). Every later
+ *    ioctl other than destroy and GEM close returns it too.
+ */
+
+/**
+ * PVR_SGX_IOCTL() - Build a powervr_sgx ioctl number
+ * @_ioctl: Incrementing id of the ioctl. Added to %DRM_COMMAND_BASE.
+ * @_mode: One of %DRM_IOW or %DRM_IOWR.
+ * @_data: Middle of the args struct name, drm_powervr_sgx_ioctl_<_data>_args.
+ */
+#define PVR_SGX_IOCTL(_ioctl, _mode, _data) \
+	_mode(DRM_COMMAND_BASE + (_ioctl), struct drm_powervr_sgx_ioctl_##_data##_args)
+
+#define DRM_IOCTL_POWERVR_SGX_DEV_QUERY PVR_SGX_IOCTL(0x00, DRM_IOWR, dev_query)
+#define DRM_IOCTL_POWERVR_SGX_CREATE_BO PVR_SGX_IOCTL(0x01, DRM_IOWR, create_bo)
+#define DRM_IOCTL_POWERVR_SGX_GET_BO_MMAP_OFFSET PVR_SGX_IOCTL(0x02, DRM_IOWR, get_bo_mmap_offset)
+#define DRM_IOCTL_POWERVR_SGX_CREATE_VM_CONTEXT PVR_SGX_IOCTL(0x03, DRM_IOWR, create_vm_context)
+#define DRM_IOCTL_POWERVR_SGX_DESTROY_VM_CONTEXT PVR_SGX_IOCTL(0x04, DRM_IOW, destroy_vm_context)
+#define DRM_IOCTL_POWERVR_SGX_VM_BIND PVR_SGX_IOCTL(0x05, DRM_IOW, vm_bind)
+#define DRM_IOCTL_POWERVR_SGX_CREATE_CONTEXT PVR_SGX_IOCTL(0x06, DRM_IOWR, create_context)
+#define DRM_IOCTL_POWERVR_SGX_DESTROY_CONTEXT PVR_SGX_IOCTL(0x07, DRM_IOW, destroy_context)
+#define DRM_IOCTL_POWERVR_SGX_GET_CONTEXT_STATE PVR_SGX_IOCTL(0x08, DRM_IOWR, get_context_state)
+#define DRM_IOCTL_POWERVR_SGX_CREATE_RT_DATASET PVR_SGX_IOCTL(0x09, DRM_IOWR, create_rt_dataset)
+#define DRM_IOCTL_POWERVR_SGX_DESTROY_RT_DATASET PVR_SGX_IOCTL(0x0a, DRM_IOW, destroy_rt_dataset)
+#define DRM_IOCTL_POWERVR_SGX_UPLOAD_CODE PVR_SGX_IOCTL(0x0b, DRM_IOWR, upload_code)
+#define DRM_IOCTL_POWERVR_SGX_SUBMIT_JOBS PVR_SGX_IOCTL(0x0c, DRM_IOWR, submit_jobs)
+
+/**
+ * DOC: PowerVR SGX IOCTL DEV_QUERY interface
+ */
+
+/**
+ * struct drm_powervr_sgx_dev_query_gpu_info - Identity of the GPU and its firmware
+ *
+ * Fetched with %DRM_POWERVR_SGX_DEV_QUERY_GPU_INFO_GET.
+ */
+struct drm_powervr_sgx_dev_query_gpu_info {
+	/** @core_id: EUR_CR_CORE_ID as read from the hardware. */
+	__u32 core_id;
+
+	/**
+	 * @core_revision: EUR_CR_CORE_REVISION as read from the hardware.
+	 * 0x1000e on the CE5300's revision 1.0.14.
+	 */
+	__u32 core_revision;
+
+	/**
+	 * @fw_interface_version: Host interface of the loaded firmware, one of
+	 * ``DRM_POWERVR_SGX_FW_IF_*``.
+	 */
+	__u32 fw_interface_version;
+
+	/** @_padding_c: Reserved. This field must be zero. */
+	__u32 _padding_c;
+};
+
+/*
+ * Version 1 of the open microkernel's own host interface: its command buffer,
+ * control blocks and command layouts.
+ */
+#define DRM_POWERVR_SGX_FW_IF_V1 1
+
+/**
+ * struct drm_powervr_sgx_dev_query_limits - Limits of the object-creating ioctls
+ *
+ * Fetched with %DRM_POWERVR_SGX_DEV_QUERY_LIMITS_GET.
+ */
+struct drm_powervr_sgx_dev_query_limits {
+	/** @rt_max_width: Largest &drm_powervr_sgx_ioctl_create_rt_dataset_args.width. */
+	__u32 rt_max_width;
+
+	/** @rt_max_height: Largest &drm_powervr_sgx_ioctl_create_rt_dataset_args.height. */
+	__u32 rt_max_height;
+
+	/** @rt_max_samples: Largest &drm_powervr_sgx_ioctl_create_rt_dataset_args.samples. */
+	__u32 rt_max_samples;
+
+	/**
+	 * @pb_min_pages: Smallest
+	 * &drm_powervr_sgx_ioctl_create_context_args.param_buffer_pages.
+	 */
+	__u32 pb_min_pages;
+
+	/**
+	 * @pb_default_pages: Size used when
+	 * &drm_powervr_sgx_ioctl_create_context_args.param_buffer_pages is 0.
+	 */
+	__u32 pb_default_pages;
+
+	/**
+	 * @pb_max_pages: Largest
+	 * &drm_powervr_sgx_ioctl_create_context_args.param_buffer_pages.
+	 */
+	__u32 pb_max_pages;
+
+	/** @max_ta_kicks: Largest &drm_powervr_sgx_job.ta_kicks count. */
+	__u32 max_ta_kicks;
+
+	/**
+	 * @va_bits: Device-virtual address bits: every address a command names
+	 * is below 2^va_bits.
+	 */
+	__u32 va_bits;
+
+	/** @caps: ``DRM_POWERVR_SGX_CAP_*`` bits of this core. */
+	__u32 caps;
+
+	/** @max_layers: Largest number of render-target layers. */
+	__u32 max_layers;
+};
+
+/**
+ * DRM_POWERVR_SGX_CAP_BACKWARD_BLITS - A transfer may copy between overlapping
+ * ranges with the destination above the source.
+ */
+#define DRM_POWERVR_SGX_CAP_BACKWARD_BLITS _BITUL(0)
+
+/**
+ * struct drm_powervr_sgx_dev_query_quirks - Hardware fixes userspace has to know
+ *
+ * Fetched with %DRM_POWERVR_SGX_DEV_QUERY_QUIRKS_GET. A quirk, once reported
+ * for a GPU, is never removed for it.
+ */
+struct drm_powervr_sgx_dev_query_quirks {
+	/**
+	 * @quirks: User pointer to a __u32 array of quirk numbers. The first
+	 * @musthave_count entries are mandatory: a client that does not
+	 * implement them must fail to initialize. The rest are opt-in.
+	 */
+	__u64 quirks;
+
+	/** @count: Length of @quirks. Updated with the number of quirks known. */
+	__u16 count;
+
+	/** @musthave_count: Number of mandatory entries at the start of @quirks. */
+	__u16 musthave_count;
+
+	/** @_padding_c: Reserved. This field must be zero. */
+	__u32 _padding_c;
+};
+
+/**
+ * enum drm_powervr_sgx_heap_id - Index into the array returned by
+ * %DRM_POWERVR_SGX_DEV_QUERY_HEAP_INFO_GET
+ *
+ * Append only. A heap a device does not have is returned with a size of zero.
+ */
+enum drm_powervr_sgx_heap_id {
+	/** @DRM_POWERVR_SGX_HEAP_GENERAL: Vertices, indices, VDM streams, surfaces. */
+	DRM_POWERVR_SGX_HEAP_GENERAL = 0,
+	/** @DRM_POWERVR_SGX_HEAP_PDS_PIXEL: PDS programs referenced from state words. */
+	DRM_POWERVR_SGX_HEAP_PDS_PIXEL,
+	/** @DRM_POWERVR_SGX_HEAP_PDS_VERTEX: PDS programs referenced from VDM streams. */
+	DRM_POWERVR_SGX_HEAP_PDS_VERTEX,
+	/** @DRM_POWERVR_SGX_HEAP_USSE_PIXEL: USSE code run by pixel tasks. */
+	DRM_POWERVR_SGX_HEAP_USSE_PIXEL,
+	/** @DRM_POWERVR_SGX_HEAP_USSE_VERTEX: USSE code run by vertex and state tasks. */
+	DRM_POWERVR_SGX_HEAP_USSE_VERTEX,
+	/** @DRM_POWERVR_SGX_HEAP_TEXTURE: Texels. */
+	DRM_POWERVR_SGX_HEAP_TEXTURE,
+	/** @DRM_POWERVR_SGX_HEAP_COUNT: Number of heaps known to this header. */
+	DRM_POWERVR_SGX_HEAP_COUNT
+};
+
+/**
+ * struct drm_powervr_sgx_heap - One device-virtual heap
+ *
+ * Heaps are the same in every VM context of a device.
+ */
+struct drm_powervr_sgx_heap {
+	/** @base: Device-virtual address of the heap. */
+	__u64 base;
+
+	/** @size: Size in bytes the client can use. 0 if the heap is not present. */
+	__u64 size;
+
+	/** @flags: Always 0. */
+	__u32 flags;
+
+	/** @page_size_log2: Log2 of the device page size. */
+	__u32 page_size_log2;
+
+	/**
+	 * @usse_base_index: First EUR_CR_USE_CODE_BASE register covering this
+	 * heap, for the DOUTU words a client writes. 0 for a non-USSE heap.
+	 */
+	__u32 usse_base_index;
+
+	/**
+	 * @usse_window_log2: Log2 of the window each code base covers. 0 for a
+	 * non-USSE heap.
+	 */
+	__u32 usse_window_log2;
+};
+
+/**
+ * struct drm_powervr_sgx_dev_query_heap_info - Array of heaps
+ *
+ * Fetched with %DRM_POWERVR_SGX_DEV_QUERY_HEAP_INFO_GET.
+ */
+struct drm_powervr_sgx_dev_query_heap_info {
+	/**
+	 * @heaps: Array of &struct drm_powervr_sgx_heap, indexed by &enum
+	 * drm_powervr_sgx_heap_id. With a NULL array pointer, the stride and count
+	 * are set to what the kernel knows and nothing else is written.
+	 */
+	struct drm_powervr_sgx_obj_array heaps;
+};
+
+/**
+ * enum drm_powervr_sgx_dev_query - Values of &drm_powervr_sgx_ioctl_dev_query_args.type
+ *
+ * Append only.
+ */
+enum drm_powervr_sgx_dev_query {
+	/** @DRM_POWERVR_SGX_DEV_QUERY_GPU_INFO_GET: &struct drm_powervr_sgx_dev_query_gpu_info. */
+	DRM_POWERVR_SGX_DEV_QUERY_GPU_INFO_GET = 0,
+	/** @DRM_POWERVR_SGX_DEV_QUERY_LIMITS_GET: &struct drm_powervr_sgx_dev_query_limits. */
+	DRM_POWERVR_SGX_DEV_QUERY_LIMITS_GET,
+	/** @DRM_POWERVR_SGX_DEV_QUERY_QUIRKS_GET: &struct drm_powervr_sgx_dev_query_quirks. */
+	DRM_POWERVR_SGX_DEV_QUERY_QUIRKS_GET,
+	/**
+	 * @DRM_POWERVR_SGX_DEV_QUERY_HEAP_INFO_GET: &struct drm_powervr_sgx_dev_query_heap_info.
+	 */
+	DRM_POWERVR_SGX_DEV_QUERY_HEAP_INFO_GET,
+	/**
+	 * @DRM_POWERVR_SGX_DEV_QUERY_COMMANDS_GET: &struct drm_powervr_sgx_dev_query_commands.
+	 */
+	DRM_POWERVR_SGX_DEV_QUERY_COMMANDS_GET,
+};
+
+/**
+ * struct drm_powervr_sgx_dev_query_commands - What this core accepts
+ *
+ * Fetched with %DRM_POWERVR_SGX_DEV_QUERY_COMMANDS_GET. A job type whose bit is
+ * clear is refused with -EOPNOTSUPP: a value it depends on is not known for this
+ * core yet. An extension name outside its command's mask is refused the same way.
+ */
+struct drm_powervr_sgx_dev_query_commands {
+	/** @job_types: Bit n: &enum drm_powervr_sgx_job_type n is accepted. */
+	__u32 job_types;
+
+	/** @_padding_4: Reserved. This field must be zero. */
+	__u32 _padding_4;
+
+	/** @render_exts: Bit n: extension name n is accepted in a render command. */
+	__u64 render_exts;
+
+	/** @ta_kick_exts: Bit n: extension name n is accepted in a TA kick. */
+	__u64 ta_kick_exts;
+
+	/** @transfer_exts: Bit n: extension name n is accepted in a transfer command. */
+	__u64 transfer_exts;
+};
+
+/**
+ * struct drm_powervr_sgx_ioctl_dev_query_args - Arguments for %DRM_IOCTL_POWERVR_SGX_DEV_QUERY
+ */
+struct drm_powervr_sgx_ioctl_dev_query_args {
+	/** @type: [IN] Query, one of &enum drm_powervr_sgx_dev_query. */
+	__u32 type;
+
+	/**
+	 * @size: [IN/OUT] Size of the receiving struct. Set to the bytes
+	 * written. With a NULL @pointer, set to the size the kernel knows.
+	 */
+	__u32 size;
+
+	/** @pointer: [IN] User pointer to the receiving struct. */
+	__u64 pointer;
+};
+
+/**
+ * DOC: PowerVR SGX IOCTL CREATE_BO interface
+ *
+ * Buffer objects are GEM objects backed by shmem and zeroed at allocation. PRIME
+ * export and import use the DRM core ioctls.
+ */
+
+/**
+ * DRM_POWERVR_SGX_BO_ALLOW_CPU_ACCESS - Allow %DRM_IOCTL_POWERVR_SGX_GET_BO_MMAP_OFFSET
+ * on this object.
+ */
+#define DRM_POWERVR_SGX_BO_ALLOW_CPU_ACCESS _BITULL(0)
+
+/**
+ * DRM_POWERVR_SGX_BO_CODE - A code object: USSE programs.
+ *
+ * Only code objects can be mapped into the USSE heaps, and a code object can be
+ * mapped nowhere else. The GPU sees it read-only. It has no CPU mapping and
+ * can't be exported, so %DRM_POWERVR_SGX_BO_ALLOW_CPU_ACCESS must be clear. Its
+ * contents arrive only through %DRM_IOCTL_POWERVR_SGX_UPLOAD_CODE, which
+ * validates them. Bits 2..63 of the flags are reserved.
+ */
+#define DRM_POWERVR_SGX_BO_CODE _BITULL(1)
+
+/**
+ * struct drm_powervr_sgx_ioctl_create_bo_args - Arguments for %DRM_IOCTL_POWERVR_SGX_CREATE_BO
+ */
+struct drm_powervr_sgx_ioctl_create_bo_args {
+	/** @size: [IN] Size in bytes. Must be page aligned and not zero. */
+	__u64 size;
+
+	/** @handle: [OUT] GEM handle of the new object. */
+	__u32 handle;
+
+	/** @_padding_c: Reserved. This field must be zero. */
+	__u32 _padding_c;
+
+	/** @flags: [IN] ``DRM_POWERVR_SGX_BO_*`` flags. */
+	__u64 flags;
+};
+
+/**
+ * struct drm_powervr_sgx_ioctl_get_bo_mmap_offset_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_GET_BO_MMAP_OFFSET
+ */
+struct drm_powervr_sgx_ioctl_get_bo_mmap_offset_args {
+	/** @handle: [IN] GEM handle. */
+	__u32 handle;
+
+	/** @_padding_4: Reserved. This field must be zero. */
+	__u32 _padding_4;
+
+	/** @offset: [OUT] Fake offset to pass to mmap() on the DRM file. */
+	__u64 offset;
+};
+
+/**
+ * DOC: PowerVR SGX IOCTL UPLOAD_CODE interface
+ *
+ * Copies USSE instructions into a code object and validates them
+ * (Documentation/gpu/powervr_sgx, "Shader validation"). An instruction is
+ * rejected if:
+ *
+ *  - it is not exactly one of the forms the kernel knows;
+ *  - it is a form the kernel refuses: register-file access (``ldr``, ``str``),
+ *    the link address (``setl``, ``savl``, ``lapc``, a branch with ``link``),
+ *    ``ba``, ``phas`` with a register operand, ``lock``, ``release``, the
+ *    operand-stepping controls other than ``smlsi``, ``fdual``, and forms
+ *    whose meaning is not known;
+ *  - it is a ``br`` whose target is outside the object;
+ *  - it is a ``phas.i`` with a target other than 0 that is outside the object,
+ *    counting from the base of the USSE heap the object is mapped in, or the
+ *    object is not mapped, or mapped at more than one place;
+ *  - an operand writes a constant or global, reads a global past ``g45``, or
+ *    names an internal register past ``i3`` or an index register other than
+ *    ``idx1`` and ``idx2``.
+ *
+ * Once an object holds code with ``phas.i`` targets, a VM_BIND map that would
+ * put it anywhere else fails with -EINVAL.
+ *
+ * On a rejection nothing is copied, the ioctl fails with -EINVAL, and
+ * @bad_offset is the byte offset in the object of the first rejected
+ * instruction.
+ *
+ * Bytes no upload wrote are zero, which is a harmless instruction. While a VM
+ * maps the object an upload may only write past every byte an earlier upload
+ * wrote, so code the GPU may be running never changes under it; otherwise
+ * -EBUSY. Once no VM maps the object, the GPU has finished every job that
+ * could use those mappings, and a later command has dropped the GPU's cached
+ * walks of them, it can be uploaded anywhere again; until then a lower write
+ * fails with -EBUSY.
+ */
+
+/**
+ * struct drm_powervr_sgx_ioctl_upload_code_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_UPLOAD_CODE
+ */
+struct drm_powervr_sgx_ioctl_upload_code_args {
+	/** @handle: [IN] GEM handle of a code object (%DRM_POWERVR_SGX_BO_CODE). */
+	__u32 handle;
+
+	/** @flags: [IN] Reserved. Must be zero. */
+	__u32 flags;
+
+	/** @offset: [IN] Byte offset in the object. Must be a multiple of 8. */
+	__u64 offset;
+
+	/** @size: [IN] Bytes to copy. Must be a multiple of 8 and not zero. */
+	__u64 size;
+
+	/** @pointer: [IN] User pointer to the instructions, low word first. */
+	__u64 pointer;
+
+	/**
+	 * @bad_offset: [OUT] When validation rejects the code, the object offset
+	 * of the first rejected instruction; otherwise all ones.
+	 */
+	__u64 bad_offset;
+};
+
+/**
+ * DOC: PowerVR SGX IOCTL VM context interfaces
+ *
+ * A VM context is one GPU page directory, managed with drm_gpuvm. The client
+ * manages the address space: it picks the address of each mapping inside the
+ * heaps %DRM_POWERVR_SGX_DEV_QUERY_HEAP_INFO_GET returns. Addresses outside them
+ * cannot be mapped: the kernel keeps the rest of the 4 GiB space for itself.
+ */
+
+/**
+ * struct drm_powervr_sgx_ioctl_create_vm_context_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_CREATE_VM_CONTEXT
+ */
+struct drm_powervr_sgx_ioctl_create_vm_context_args {
+	/** @handle: [OUT] Handle of the new VM context. */
+	__u32 handle;
+
+	/** @_padding_4: Reserved. This field must be zero. */
+	__u32 _padding_4;
+};
+
+/**
+ * struct drm_powervr_sgx_ioctl_destroy_vm_context_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_DESTROY_VM_CONTEXT
+ *
+ * The handle goes at once. Once no context uses the VM either and every job on
+ * it is done, its mappings go and their objects are released; the ioctl
+ * doesn't wait for that.
+ */
+struct drm_powervr_sgx_ioctl_destroy_vm_context_args {
+	/** @handle: [IN] Handle of the VM context. */
+	__u32 handle;
+
+	/** @_padding_4: Reserved. This field must be zero. */
+	__u32 _padding_4;
+};
+
+/**
+ * DOC: VM_BIND operations
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_VM_BIND_OP_MAP
+ *
+ *    Map @size bytes of the object @handle, from @bo_offset, at @device_addr.
+ *    The range must lie inside one heap, page aligned, and must not overlap a
+ *    mapping (-EEXIST). A code object maps only into a USSE heap, and only a
+ *    code object does.
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_VM_BIND_OP_UNMAP
+ *
+ *    Unmap every mapping in [@device_addr, @device_addr + @size), splitting a
+ *    mapping that straddles either end. @handle and @bo_offset must be 0. A
+ *    range with no mapping in it is not an error.
+ *
+ * The page tables of an unmapped range change before the ioctl returns, so a
+ * job that is still running and uses the range faults inside this VM. The
+ * objects stay allocated until every job submitted before the unmap is done.
+ */
+#define DRM_POWERVR_SGX_VM_BIND_OP_MAP 0
+#define DRM_POWERVR_SGX_VM_BIND_OP_UNMAP 1
+
+/**
+ * struct drm_powervr_sgx_vm_bind_op - One map or unmap of %DRM_IOCTL_POWERVR_SGX_VM_BIND
+ */
+struct drm_powervr_sgx_vm_bind_op {
+	/** @op: [IN] ``DRM_POWERVR_SGX_VM_BIND_OP_*``. */
+	__u32 op;
+
+	/** @handle: [IN] GEM handle of the object to map; 0 for an unmap. */
+	__u32 handle;
+
+	/** @bo_offset: [IN] Offset into the object, in bytes; 0 for an unmap. */
+	__u64 bo_offset;
+
+	/** @device_addr: [IN] Device-virtual start of the range. */
+	__u64 device_addr;
+
+	/** @size: [IN] Size of the range, in bytes. */
+	__u64 size;
+
+	/** @flags: [IN] Reserved, must be 0. */
+	__u32 flags;
+
+	/** @_padding_24: Reserved. This field must be zero. */
+	__u32 _padding_24;
+};
+
+/**
+ * DOC: Flags for struct drm_powervr_sgx_ioctl_vm_bind_args
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_VM_BIND_FLAG_SYNC
+ *
+ *    The ioctl binds and returns when the binds are done. A synchronous bind
+ *    takes no fences: @sync_ops must be empty. Required: a bind without it
+ *    fails with -EINVAL.
+ */
+#define DRM_POWERVR_SGX_VM_BIND_FLAG_SYNC _BITUL(0)
+
+/**
+ * struct drm_powervr_sgx_ioctl_vm_bind_args - Arguments for %DRM_IOCTL_POWERVR_SGX_VM_BIND
+ *
+ * The ops apply in array order, and all or none of them apply: every op is
+ * checked, and everything it needs allocated, before any mapping changes. The
+ * ranges of the ops in one call must not overlap each other (-EINVAL). A wait
+ * that is interrupted fails with -EINTR before anything changes.
+ */
+struct drm_powervr_sgx_ioctl_vm_bind_args {
+	/** @vm_context_handle: [IN] VM context to bind in. */
+	__u32 vm_context_handle;
+
+	/** @flags: [IN] ``DRM_POWERVR_SGX_VM_BIND_FLAG_*``. */
+	__u32 flags;
+
+	/** @ops: [IN] Array of &struct drm_powervr_sgx_vm_bind_op, at least one. */
+	struct drm_powervr_sgx_obj_array ops;
+
+	/**
+	 * @sync_ops: [IN] Must be empty (count 0) with
+	 * %DRM_POWERVR_SGX_VM_BIND_FLAG_SYNC, the only mode. Reserved for an
+	 * asynchronous mode's &struct drm_powervr_sgx_sync_op array.
+	 */
+	struct drm_powervr_sgx_obj_array sync_ops;
+};
+
+/**
+ * DOC: PowerVR SGX IOCTL CREATE_CONTEXT interface
+ *
+ * A context is one scheduling queue on the GPU. Its firmware-visible state (HW
+ * context, command ring, parameter buffer, free list) belongs to the kernel and
+ * is never mapped into the client's address space for CPU access.
+ */
+
+/**
+ * enum drm_powervr_sgx_ctx_type - Values of &drm_powervr_sgx_ioctl_create_context_args.type
+ */
+enum drm_powervr_sgx_ctx_type {
+	/** @DRM_POWERVR_SGX_CTX_TYPE_RENDER: Runs %DRM_POWERVR_SGX_JOB_TYPE_RENDER jobs. */
+	DRM_POWERVR_SGX_CTX_TYPE_RENDER = 0,
+	/** @DRM_POWERVR_SGX_CTX_TYPE_TRANSFER: Runs %DRM_POWERVR_SGX_JOB_TYPE_TRANSFER jobs. */
+	DRM_POWERVR_SGX_CTX_TYPE_TRANSFER,
+};
+
+/**
+ * enum drm_powervr_sgx_ctx_priority - Values of
+ * &drm_powervr_sgx_ioctl_create_context_args.priority
+ */
+enum drm_powervr_sgx_ctx_priority {
+	/** @DRM_POWERVR_SGX_CTX_PRIORITY_LOW: Below normal. */
+	DRM_POWERVR_SGX_CTX_PRIORITY_LOW = -512,
+	/** @DRM_POWERVR_SGX_CTX_PRIORITY_NORMAL: Normal. */
+	DRM_POWERVR_SGX_CTX_PRIORITY_NORMAL = 0,
+	/** @DRM_POWERVR_SGX_CTX_PRIORITY_HIGH: Above normal. Requires CAP_SYS_NICE. */
+	DRM_POWERVR_SGX_CTX_PRIORITY_HIGH = 512,
+};
+
+/**
+ * struct drm_powervr_sgx_ioctl_create_context_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_CREATE_CONTEXT
+ */
+struct drm_powervr_sgx_ioctl_create_context_args {
+	/** @type: [IN] One of &enum drm_powervr_sgx_ctx_type. */
+	__u32 type;
+
+	/** @flags: [IN] Reserved, must be 0. */
+	__u32 flags;
+
+	/** @priority: [IN] One of &enum drm_powervr_sgx_ctx_priority. */
+	__s32 priority;
+
+	/** @vm_context_handle: [IN] VM context whose page directory the jobs use. */
+	__u32 vm_context_handle;
+
+	/**
+	 * @param_buffer_pages: [IN] Parameter buffer size for a render context,
+	 * in device pages, or 0 for the default in
+	 * &drm_powervr_sgx_dev_query_limits.pb_default_pages. Must be 0 for a transfer
+	 * context. The device has one parameter buffer, shared by every render
+	 * context of every client: the first render context sets its size, and
+	 * a later non-zero request for another size fails with -EINVAL.
+	 */
+	__u32 param_buffer_pages;
+
+	/** @handle: [OUT] Handle of the new context. */
+	__u32 handle;
+};
+
+/**
+ * struct drm_powervr_sgx_ioctl_destroy_context_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_DESTROY_CONTEXT
+ */
+struct drm_powervr_sgx_ioctl_destroy_context_args {
+	/** @handle: [IN] Handle of the context. */
+	__u32 handle;
+
+	/** @_padding_4: Reserved. This field must be zero. */
+	__u32 _padding_4;
+};
+
+/**
+ * enum drm_powervr_sgx_context_state - Values of
+ * &drm_powervr_sgx_ioctl_get_context_state_args.state
+ *
+ * These correspond to the reset statuses of GL_ARB_robustness.
+ */
+enum drm_powervr_sgx_context_state {
+	/** @DRM_POWERVR_SGX_CONTEXT_STATE_OK: No reset has affected this context. */
+	DRM_POWERVR_SGX_CONTEXT_STATE_OK = 0,
+	/**
+	 * @DRM_POWERVR_SGX_CONTEXT_STATE_GUILTY: A job of this context hung the GPU
+	 * or faulted. Its fence carries -ETIME or -EFAULT.
+	 */
+	DRM_POWERVR_SGX_CONTEXT_STATE_GUILTY,
+	/**
+	 * @DRM_POWERVR_SGX_CONTEXT_STATE_INNOCENT: A job of this context was running
+	 * when another context's job reset the GPU. Its fence carries -ECANCELED.
+	 */
+	DRM_POWERVR_SGX_CONTEXT_STATE_INNOCENT,
+};
+
+/**
+ * struct drm_powervr_sgx_ioctl_get_context_state_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_GET_CONTEXT_STATE
+ *
+ * A context that is not %DRM_POWERVR_SGX_CONTEXT_STATE_OK fails every later submission
+ * with -ECANCELED; the client destroys it and creates a new one. A context whose
+ * jobs were only queued during a reset stays OK and its jobs run afterwards.
+ */
+struct drm_powervr_sgx_ioctl_get_context_state_args {
+	/** @handle: [IN] Handle of the context. */
+	__u32 handle;
+
+	/** @state: [OUT] One of &enum drm_powervr_sgx_context_state. */
+	__u32 state;
+
+	/**
+	 * @reset_count: [OUT] Number of GPU resets since the driver bound the
+	 * device, for every context alike; it only grows. Compare two reads to see
+	 * whether a reset happened in between. @state says whether this context
+	 * was to blame.
+	 */
+	__u32 reset_count;
+
+	/** @_padding_c: Reserved. This field must be zero. */
+	__u32 _padding_c;
+};
+
+/**
+ * DOC: PowerVR SGX IOCTL CREATE_RT_DATASET interface
+ *
+ * An RT dataset is what tiling a target of a given size needs: the macrotile
+ * numbers, region headers, tail pointers, background object block and firmware
+ * render data. The kernel derives all of it from the size, allocates it and keeps
+ * it. A client sees only the handle.
+ */
+
+/**
+ * struct drm_powervr_sgx_ioctl_create_rt_dataset_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_CREATE_RT_DATASET
+ */
+struct drm_powervr_sgx_ioctl_create_rt_dataset_args {
+	/** @context_handle: [IN] Render context that uses this dataset. */
+	__u32 context_handle;
+
+	/** @width: [IN] Target width in pixels. */
+	__u32 width;
+
+	/** @height: [IN] Target height in pixels. */
+	__u32 height;
+
+	/**
+	 * @samples: [IN] Samples per pixel: 1 or 4 (0 is 1). 4 needs
+	 * &drm_powervr_sgx_dev_query_limits.rt_max_samples of 4 on this core;
+	 * otherwise it is -EOPNOTSUPP. A 4-sample target's renders need a
+	 * depth buffer and ZLS control, with depth and stencil per sample.
+	 */
+	__u32 samples;
+
+	/** @flags: [IN] Reserved, must be 0. */
+	__u32 flags;
+
+	/** @handle: [OUT] Handle of the new dataset. */
+	__u32 handle;
+};
+
+/**
+ * struct drm_powervr_sgx_ioctl_destroy_rt_dataset_args - Arguments for
+ * %DRM_IOCTL_POWERVR_SGX_DESTROY_RT_DATASET
+ */
+struct drm_powervr_sgx_ioctl_destroy_rt_dataset_args {
+	/** @handle: [IN] Handle of the dataset. */
+	__u32 handle;
+
+	/** @_padding_4: Reserved. This field must be zero. */
+	__u32 _padding_4;
+};
+
+/**
+ * DOC: PowerVR SGX client commands
+ *
+ * The client's part of a GPU command is one struct per command: &struct
+ * drm_powervr_sgx_cmd_render, &struct drm_powervr_sgx_cmd_ta_kick and &struct
+ * drm_powervr_sgx_cmd_transfer. Each starts with &struct drm_powervr_sgx_cmd_head:
+ * its size, and a chain of core-specific extensions. The kernel copies the
+ * command and its extensions, checks every field and builds the firmware command
+ * from the result.
+ *
+ * Addresses are device-virtual addresses in the job's VM. A field the kernel
+ * does not check is one whose value only reaches a hardware register, with no
+ * effect outside the VM. The structs below and the kernel's check tables are
+ * generated by gen_cmd.py from one description of the fields and their checks,
+ * powervr_sgx_cmd.xml; both live in the driver's source repository,
+ * https://github.com/The1TrueJoe/powervr_sgx.
+ */
+
+/* BEGIN commands: generated from powervr_sgx_cmd.xml */
+
+/**
+ * struct drm_powervr_sgx_cmd_head - The head of every command struct
+ *
+ * The kernel copies a command as copy_struct_from_user() does: a shorter
+ * struct is zero-filled, non-zero bytes past what the kernel knows are
+ * -E2BIG. A field appended later takes 0 to mean "absent".
+ */
+struct drm_powervr_sgx_cmd_head {
+	/** @size: sizeof() the struct as the caller built it. */
+	__u32 size;
+
+	/** @_padding_4: Reserved. This field must be zero. */
+	__u32 _padding_4;
+
+	/**
+	 * @extensions: User pointer to the first &struct drm_powervr_sgx_ext,
+	 * or 0.
+	 */
+	__u64 extensions;
+};
+
+/**
+ * struct drm_powervr_sgx_ext - The head of every command extension
+ *
+ * At most 16 in a chain, each name once. A name of another command is
+ * -EINVAL; a name this core doesn't accept is -EOPNOTSUPP
+ * (%DRM_POWERVR_SGX_DEV_QUERY_COMMANDS_GET lists the accepted ones).
+ */
+struct drm_powervr_sgx_ext {
+	/** @name: One of the ``DRM_POWERVR_SGX_EXT_*`` names. */
+	__u32 name;
+
+	/** @_padding_4: Reserved. This field must be zero. */
+	__u32 _padding_4;
+
+	/** @next: User pointer to the next extension, or 0. */
+	__u64 next;
+};
+
+/* &struct drm_powervr_sgx_ext_sgx545_render, for sgx545 */
+#define DRM_POWERVR_SGX_EXT_SGX545_RENDER 1
+/* &struct drm_powervr_sgx_ext_sgx545_ta_kick, for sgx545 */
+#define DRM_POWERVR_SGX_EXT_SGX545_TA_KICK 2
+
+/**
+ * struct drm_powervr_sgx_cmd_render - A render command
+ *
+ * Extensions: &struct drm_powervr_sgx_ext_sgx545_render.
+ */
+struct drm_powervr_sgx_cmd_render {
+	/** @head: The common head. */
+	struct drm_powervr_sgx_cmd_head head;
+
+	/** @depth_addr: Depth buffer the ZLS loads and stores, or 0. */
+	__u64 depth_addr;
+
+	/** @stencil_addr: Stencil buffer the ZLS loads and stores, or 0. */
+	__u64 stencil_addr;
+
+	/** @pixel_event_pds_addr: PDS program run at the end of each tile (the pixel event). */
+	__u64 pixel_event_pds_addr;
+
+	/** @bg_pds_secondary_addr: Background object: secondary PDS program. */
+	__u64 bg_pds_secondary_addr;
+
+	/** @bg_pds_primary_addr: Background object: primary PDS program. */
+	__u64 bg_pds_primary_addr;
+
+	/** @pixel_event_pds_rows: Data rows of the pixel event program. */
+	__u32 pixel_event_pds_rows;
+
+	/** @bg_pds_secondary_rows: Data rows of the background secondary program. */
+	__u32 bg_pds_secondary_rows;
+
+	/** @bg_pds_primary_rows: Data rows of the background primary program. */
+	__u32 bg_pds_primary_rows;
+
+	/**
+	 * @bg_depth_f32: Depth the background object clears to. IEEE-754
+	 * binary32 bits, any value.
+	 */
+	__u32 bg_depth_f32;
+
+	/** @depth_bias_units: Constant depth bias (glPolygonOffset units). */
+	__s32 depth_bias_units;
+
+	/**
+	 * @depth_bias_factor_f32: Slope-scaled depth bias (glPolygonOffset
+	 * factor). IEEE-754 binary32 bits, any value.
+	 */
+	__u32 depth_bias_factor_f32;
+};
+
+/**
+ * struct drm_powervr_sgx_cmd_ta_kick - A ta kick command
+ *
+ * Extensions: &struct drm_powervr_sgx_ext_sgx545_ta_kick.
+ */
+struct drm_powervr_sgx_cmd_ta_kick {
+	/** @head: The common head. */
+	struct drm_powervr_sgx_cmd_head head;
+
+	/** @vdm_base: Start of the kick's VDM control stream. */
+	__u64 vdm_base;
+};
+
+/**
+ * struct drm_powervr_sgx_cmd_transfer - A transfer command
+ */
+struct drm_powervr_sgx_cmd_transfer {
+	/** @head: The common head. */
+	struct drm_powervr_sgx_cmd_head head;
+
+	/** @bg_block_addr: Background block: the blit's PDS program triple. */
+	__u64 bg_block_addr;
+
+	/** @pixel_event_pds_addr: PDS program run at the end of each tile. */
+	__u64 pixel_event_pds_addr;
+
+	/** @pixel_event_pds_rows: Data rows of the pixel event program. */
+	__u32 pixel_event_pds_rows;
+
+	/** @first_tile_x: First tile column to blit. */
+	__u32 first_tile_x;
+
+	/** @first_tile_y: First tile row to blit. */
+	__u32 first_tile_y;
+
+	/** @last_tile_x: Last tile column to blit. */
+	__u32 last_tile_x;
+
+	/** @last_tile_y: Last tile row to blit. */
+	__u32 last_tile_y;
+
+	/** @bg_depth_f32: Depth the background object writes. IEEE-754 binary32 bits, any value. */
+	__u32 bg_depth_f32;
+};
+
+/**
+ * struct drm_powervr_sgx_ext_sgx545_render - Extension %DRM_POWERVR_SGX_EXT_SGX545_RENDER
+ *
+ * For &struct drm_powervr_sgx_cmd_render, on sgx545.
+ * Absent, its fields read as zero.
+ */
+struct drm_powervr_sgx_ext_sgx545_render {
+	/** @base: The common head. */
+	struct drm_powervr_sgx_ext base;
+
+	/** @zls_control: ZLS control register image; bits 3-11 are the kernel's. */
+	__u32 zls_control;
+
+	/** @flags_3d: 3D flags; bit 8 uses the stencil buffer. */
+	__u32 flags_3d;
+};
+
+/**
+ * struct drm_powervr_sgx_ext_sgx545_ta_kick - Extension %DRM_POWERVR_SGX_EXT_SGX545_TA_KICK
+ *
+ * For &struct drm_powervr_sgx_cmd_ta_kick, on sgx545.
+ * Absent, its fields read as zero.
+ */
+struct drm_powervr_sgx_ext_sgx545_ta_kick {
+	/** @base: The common head. */
+	struct drm_powervr_sgx_ext base;
+
+	/** @ta_flags2: TA flags register image. */
+	__u32 ta_flags2;
+
+	/** @_padding_14: Reserved. This field must be zero. */
+	__u32 _padding_14;
+};
+
+/* END commands */
+
+/**
+ * DOC: PowerVR SGX IOCTL SUBMIT_JOBS interface
+ *
+ * The ioctl is all-or-nothing: when it fails, no job of the array was queued.
+ * Jobs on one context run in submission order. Jobs on different contexts are
+ * ordered only by the sync operations.
+ *
+ * A job has one out-fence, signalled from the completion interrupt when the whole
+ * job is done: for a render job, when its 3D has finished. A job that does not
+ * finish within the driver's timeout is killed and the GPU reset.
+ */
+
+/**
+ * DOC: Flags for struct drm_powervr_sgx_sync_op
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_SYNCOBJ
+ *
+ *    The handle is a binary syncobj. This is the default.
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_TIMELINE_SYNCOBJ
+ *
+ *    The handle is a timeline syncobj and the value names a point on it.
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_SYNC_OP_FLAG_WAIT
+ *
+ *    The job waits for this syncobj before it starts. This is the default.
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_SYNC_OP_FLAG_SIGNAL
+ *
+ *    The job's out-fence is attached to this syncobj.
+ */
+#define DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_MASK 0xfU
+#define DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_SYNCOBJ 0U
+#define DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_TIMELINE_SYNCOBJ 1U
+#define DRM_POWERVR_SGX_SYNC_OP_FLAG_WAIT 0U
+#define DRM_POWERVR_SGX_SYNC_OP_FLAG_SIGNAL _BITUL(31)
+
+/**
+ * struct drm_powervr_sgx_sync_op - One wait or signal of a job
+ */
+struct drm_powervr_sgx_sync_op {
+	/** @handle: Syncobj handle. */
+	__u32 handle;
+
+	/** @flags: ``DRM_POWERVR_SGX_SYNC_OP_FLAG_*`` flags. */
+	__u32 flags;
+
+	/** @value: Timeline point. Must be 0 for a binary syncobj. */
+	__u64 value;
+};
+
+/**
+ * DOC: Flags for struct drm_powervr_sgx_cache_range
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_CACHE_RANGE_FLAG_READ
+ *
+ *    The job reads the range.
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_CACHE_RANGE_FLAG_WRITE
+ *
+ *    The job writes the range.
+ */
+#define DRM_POWERVR_SGX_CACHE_RANGE_FLAG_READ _BITUL(0)
+#define DRM_POWERVR_SGX_CACHE_RANGE_FLAG_WRITE _BITUL(1)
+
+/**
+ * struct drm_powervr_sgx_cache_range - Bytes of an import a job reads or writes
+ *
+ * The GPU doesn't snoop the CPU caches, and an imported object's pages may be
+ * cached. The kernel keeps a job coherent with the CPU on the listed bytes
+ * only:
+ *
+ *  - when the job is about to run, it writes back and drops the CPU's lines of
+ *    every range, so the GPU reads what the CPU wrote and no dirty line lands on
+ *    the job's output later;
+ *  - once the job's writes have reached memory, and before its out-fence
+ *    signals, it drops the lines of every range with
+ *    %DRM_POWERVR_SGX_CACHE_RANGE_FLAG_WRITE, which the CPU may have fetched
+ *    again meanwhile. A job whose out-fence carries an error gets no such
+ *    flush: its WRITE ranges are undefined.
+ *
+ * The range is @rows rows of @size bytes, @stride bytes apart, from @offset,
+ * widened to whole cache lines: one damage rectangle of a linear buffer is one
+ * entry. @rows 0 is one row. An entry for an object that isn't an import (ours
+ * are write-combined, and a dma-buf of this device imports as the original
+ * object) is checked and flushes nothing. No fence is attached.
+ *
+ * Bytes of an import the job changes but doesn't list have undefined contents
+ * afterwards, for the CPU and the GPU alike. That is never a leak: the kernel
+ * writes the whole import back once when it is imported, so a stale line or a
+ * stale DRAM word is the object's own earlier contents. Bytes a job rewrites
+ * unchanged (a render's tiles outside its draws, which it loaded) need no entry
+ * while the CPU doesn't write the object during the job.
+ */
+struct drm_powervr_sgx_cache_range {
+	/** @handle: GEM handle. */
+	__u32 handle;
+
+	/** @flags: ``DRM_POWERVR_SGX_CACHE_RANGE_FLAG_*``, at least one. */
+	__u32 flags;
+
+	/** @offset: Byte offset of the first row. */
+	__u64 offset;
+
+	/** @size: Bytes per row. Must not be 0. */
+	__u64 size;
+
+	/** @stride: Bytes from one row to the next; at least @size when @rows > 1. */
+	__u32 stride;
+
+	/** @rows: Number of rows; 0 and 1 both mean one. */
+	__u32 rows;
+};
+
+/**
+ * DOC: Flags for &drm_powervr_sgx_job.flags
+ *
+ * Valid flags depend on the job type. Render jobs have none yet.
+ *
+ * .. c:macro:: DRM_POWERVR_SGX_SUBMIT_JOB_TRANSFER_DST_TWIDDLED
+ *
+ *    The transfer's destination surface is twiddled.
+ */
+#define DRM_POWERVR_SGX_SUBMIT_JOB_RENDER_FLAGS_MASK 0U
+#define DRM_POWERVR_SGX_SUBMIT_JOB_TRANSFER_DST_TWIDDLED _BITUL(0)
+#define DRM_POWERVR_SGX_SUBMIT_JOB_TRANSFER_FLAGS_MASK \
+	DRM_POWERVR_SGX_SUBMIT_JOB_TRANSFER_DST_TWIDDLED
+
+/**
+ * enum drm_powervr_sgx_job_type - Values of &drm_powervr_sgx_job.type
+ */
+enum drm_powervr_sgx_job_type {
+	/** @DRM_POWERVR_SGX_JOB_TYPE_RENDER: A whole scene: TA kicks and the 3D. */
+	DRM_POWERVR_SGX_JOB_TYPE_RENDER = 0,
+	/** @DRM_POWERVR_SGX_JOB_TYPE_TRANSFER: One 3D-only blit. */
+	DRM_POWERVR_SGX_JOB_TYPE_TRANSFER,
+	/**
+	 * @DRM_POWERVR_SGX_JOB_TYPE_NULL: No GPU work. The firmware takes it in
+	 * queue order with the context's other jobs, and its out-fences signal
+	 * once it has. Runs on a context of either type; no command, no flags,
+	 * no TA kicks, no RT dataset. For testing synchronization (and for a
+	 * fence that follows everything queued before it).
+	 */
+	DRM_POWERVR_SGX_JOB_TYPE_NULL,
+};
+
+/**
+ * struct drm_powervr_sgx_job - One job of %DRM_IOCTL_POWERVR_SGX_SUBMIT_JOBS
+ */
+struct drm_powervr_sgx_job {
+	/** @type: One of &enum drm_powervr_sgx_job_type. */
+	__u32 type;
+
+	/** @context_handle: Context of the matching type. */
+	__u32 context_handle;
+
+	/**
+	 * @flags: ``DRM_POWERVR_SGX_SUBMIT_JOB_<type>_*`` flags. Bits outside
+	 * the type's mask are rejected.
+	 */
+	__u32 flags;
+
+	/** @_padding_c: Reserved. This field must be zero. */
+	__u32 _padding_c;
+
+	/**
+	 * @cmd: User pointer to the type's command: &struct
+	 * drm_powervr_sgx_cmd_render or &struct drm_powervr_sgx_cmd_transfer.
+	 */
+	__u64 cmd;
+
+	/** @sync_ops: Array of &struct drm_powervr_sgx_sync_op. May be empty. */
+	struct drm_powervr_sgx_obj_array sync_ops;
+
+	/**
+	 * @cache_ranges: Array of &struct drm_powervr_sgx_cache_range, at most
+	 * 64: the bytes of imports the job reads or writes. May be empty.
+	 */
+	struct drm_powervr_sgx_obj_array cache_ranges;
+
+	/**
+	 * @ta_kicks: Array of &struct drm_powervr_sgx_cmd_ta_kick, each with
+	 * &drm_powervr_sgx_cmd_head.size equal to the stride. At least one
+	 * element and at most &drm_powervr_sgx_dev_query_limits.max_ta_kicks for a
+	 * render job, empty for a transfer job.
+	 */
+	struct drm_powervr_sgx_obj_array ta_kicks;
+
+	/** @rt_dataset_handle: RT dataset of a render job, 0 for a transfer job. */
+	__u32 rt_dataset_handle;
+
+	/** @_padding_4c: Reserved. This field must be zero. */
+	__u32 _padding_4c;
+};
+
+/**
+ * struct drm_powervr_sgx_ioctl_submit_jobs_args - Arguments for %DRM_IOCTL_POWERVR_SGX_SUBMIT_JOBS
+ *
+ * Returns -ECANCELED when a job's context is not usable after a reset.
+ */
+struct drm_powervr_sgx_ioctl_submit_jobs_args {
+	/** @jobs: [IN] Array of &struct drm_powervr_sgx_job. */
+	struct drm_powervr_sgx_obj_array jobs;
+
+	/**
+	 * @failed_index: [OUT] On error, the index of the job that caused it,
+	 * or ~0 when no single job did. No job was submitted.
+	 */
+	__u32 failed_index;
+
+	/** @_padding_14: Reserved. This field must be zero. */
+	__u32 _padding_14;
+};
+
+#if defined(__cplusplus)
+}
+#endif
+
+#endif /* POWERVR_SGX_DRM_H */

-- 
2.50.1

[PATCH RFC 05/13] drm/powervr_sgx: Add the core table, registers and device bring-up

[Joseph Telaak]

The first part of powervr_sgx, a render-only DRM driver for the PowerVR
SGX5xx (Series5) GPUs. Nothing builds until the Kconfig entry, in the
last driver patch of this series.

- A core table names each supported core: its ID, the registers that
  move between SGX5xx cores, features, instruction set, directory lists
  and init script, and per part its revision, clock and firmware file.
  The bus says which core to expect, and a core the table doesn't have,
  or one the driver can't run, is refused before any register write.
- Each core's heap layout, quirks and register fields come from its
  entry, and nothing writes a register a core doesn't have.
- The register names come from the GPL SGX545 headers, through a
  generated pvrsgx_regs.h.
- Kernel memory and the GPU's two-level page tables, below 4 GiB and
  write-combined in every CPU mapping.
- The reset sequence: every directory list cleared, list 0 on a scratch
  directory that absorbs leftover faults, the microkernel's directory on
  the last list.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/powervr_sgx/pvrsgx_cache.h  |  66 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_core.c   | 296 ++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_core.h   | 196 ++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_device.c | 461 ++++++++++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_device.h | 354 +++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_hw.h     |  55 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.c    | 281 +++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.h    |  28 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_regs.h   |  79 +++++
 9 files changed, 1816 insertions(+)

diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_cache.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_cache.h
new file mode 100644
index 000000000..76ccee62e
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_cache.h
@@ -0,0 +1,66 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_CACHE_H
+#define PVRSGX_CACHE_H
+
+/*
+ * The SGX doesn't snoop the CPU caches. Memory the CPU writes for the GPU is
+ * written through uncached (write-combined) mappings. On x86 the linear map
+ * of those pages is made write-combined too. Elsewhere (ARM) the linear map
+ * stays cacheable: the lines the page's zeroing left are written back once,
+ * and the CPU only writes the page through its own write-combined mapping.
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+
+#if IS_ENABLED(CONFIG_X86)
+#include <asm/set_memory.h>
+
+static inline int pvrsgx_pages_wc(struct device *dev, struct page **pages, unsigned int n)
+{
+	return set_pages_array_wc(pages, n);
+}
+
+static inline void pvrsgx_pages_wb(struct page **pages, unsigned int n)
+{
+	set_pages_array_wb(pages, n);
+}
+
+/* A page table's CPU mapping: its linear map is already write-combined. */
+static inline u32 *pvrsgx_table_map(struct page *p)
+{
+	return page_address(p);
+}
+
+static inline void pvrsgx_table_unmap(u32 *cpu) { }
+#else
+static inline int pvrsgx_pages_wc(struct device *dev, struct page **pages, unsigned int n)
+{
+	for (unsigned int i = 0; i < n; i++) {
+		/* Mapping for the device writes back the page's dirty lines. */
+		dma_addr_t a = dma_map_page(dev, pages[i], 0, PAGE_SIZE, DMA_TO_DEVICE);
+
+		if (dma_mapping_error(dev, a))
+			return -ENOMEM;
+		dma_unmap_page(dev, a, PAGE_SIZE, DMA_TO_DEVICE);
+	}
+	return 0;
+}
+
+static inline void pvrsgx_pages_wb(struct page **pages, unsigned int n) { }
+
+static inline u32 *pvrsgx_table_map(struct page *p)
+{
+	return vmap(&p, 1, VM_MAP, pgprot_writecombine(PAGE_KERNEL));
+}
+
+static inline void pvrsgx_table_unmap(u32 *cpu)
+{
+	vunmap(cpu);
+}
+#endif
+
+#endif /* PVRSGX_CACHE_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_core.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_core.c
new file mode 100644
index 000000000..dd8fd14eb
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_core.c
@@ -0,0 +1,296 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The SGX5xx cores this driver knows. A core that is not listed is refused
+ * before any register write: on an unknown core the driver can't tell which
+ * register writes are safe.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/sizes.h>
+
+#include "pvrsgx_cmd_defs.h"
+#include "pvrsgx_core.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_hw.h"
+
+bool pvrsgx_allow_unisolated;
+module_param_named(allow_unisolated, pvrsgx_allow_unisolated, bool, 0444);
+MODULE_PARM_DESC(allow_unisolated,
+		 "Run a single-context core (SGX530), where clients can reach the firmware and each other's commands (default off)");
+
+static const struct pvrsgx_core_regs sgx545_regs = {
+	.core_id = SGX_CR_CORE_ID,
+	.core_revision = SGX_CR_CORE_REVISION,
+	.clkgatectl_units = SGX_CR_CLKGATECTL_UNITS,
+	.clkgatectl_dpm = SGX_CR_CLKGATECTL_DPM_CLKG,
+	.clkgatectl2 = SGX_CR_CLKGATECTL2,
+	.clkgatectl2_units = SGX_CR_CLKGATECTL2_UNITS,
+	.clkgatestatus = SGX_CR_CLKGATESTATUS,
+	.clkgatestatus_dpm = SGX_CR_CLKGATESTATUS_DPM_CLKS,
+	.soft_reset_bif = SGX_CR_SOFT_RESET_BIF_RESET,
+	.soft_reset_cores = SGX_CR_SOFT_RESET_DPM_RESET | SGX_CR_SOFT_RESET_TA_RESET |
+			    SGX_CR_SOFT_RESET_USE_RESET | SGX_CR_SOFT_RESET_ISP_RESET |
+			    SGX_CR_SOFT_RESET_TSP_RESET,
+	.event_kick2 = SGX_CR_EVENT_KICK2,
+	.event_kick2_now = SGX_CR_EVENT_KICK2_NOW,
+	.use_code_base0 = SGX_CR_USE_CODE_BASE_0,
+	.pds_exec_base = SGX_CR_PDS_EXEC_BASE,
+	.bif_int_stat_fault_req = SGX_CR_BIF_INT_STAT_FAULT_REQ,
+	.bif_int_stat_fault_type = SGX_CR_BIF_INT_STAT_FAULT_TYPE,
+	.bif_fault_addr = SGX_MMU_ADDR_MASK,
+	.bif_bank_set = SGX_CR_BIF_BANK_SET,
+	.bif_bank0 = SGX_CR_BIF_BANK0,
+	.use_code_base_dm_shift = 25,
+	.use_code_base_addr_shift = 7,
+	.cr_b30 = 0x0b30,
+	.event_pds_enable = SGX545_CR_EVENT_PDS_ENABLE,
+	.event_pds_enable_all = SGX545_CR_EVENT_PDS_ENABLE_ALL,
+	.event_pds_base = SGX545_CR_EVENT_PDS_BASE,
+	.event_pds_data0 = SGX545_CR_EVENT_PDS_DATA0,
+	.event_pds_data1 = SGX545_CR_EVENT_PDS_DATA1,
+	.edm_task_reg0 = SGX545_CR_EDM_TASK_REG0,
+	.edm_task_reg1 = SGX545_CR_EDM_TASK_REG1,
+};
+
+/* Each heap runs to the next one's base. */
+static const struct pvrsgx_heap_layout sgx545_heaps = {
+	.params_3d = { PVRSGX_HEAP_3D_PARAMS, PVRSGX_HEAP_TA_DATA },
+	.ta_data = { PVRSGX_HEAP_TA_DATA, PVRSGX_HEAP_SYNC },
+	.sync = { PVRSGX_HEAP_SYNC, PVRSGX_HEAP_PDS_PIXEL },
+	.pds_pixel = { PVRSGX_HEAP_PDS_PIXEL, PVRSGX_HEAP_KERNEL_CODE },
+	.kernel_code = { PVRSGX_HEAP_KERNEL_CODE, PVRSGX_HEAP_PDS_VERTEX },
+	.pds_vertex = { PVRSGX_HEAP_PDS_VERTEX, PVRSGX_HEAP_KERNEL_DATA },
+	.kernel_data = { PVRSGX_HEAP_KERNEL_DATA, PVRSGX_HEAP_USSE_PIXEL },
+	.usse_pixel = { PVRSGX_HEAP_USSE_PIXEL,
+			PVRSGX_HEAP_USSE_PIXEL + PVRSGX_HEAP_USSE_PIXEL_SIZE },
+	.usse_vertex = { PVRSGX_HEAP_USSE_VERTEX,
+			 PVRSGX_HEAP_USSE_VERTEX + PVRSGX_HEAP_USSE_VERTEX_SIZE },
+	.general = { PVRSGX_HEAP_GENERAL, PVRSGX_HEAP_TEXTURE },
+	.texture = { PVRSGX_HEAP_TEXTURE, SZ_4G },
+	.pb_max_pages = 0x8000,
+};
+
+/*
+ * Init script part 2, written in this order. Most of these registers have no
+ * name in the GPL header.
+ */
+static const struct pvrsgx_core_init_reg sgx545_init_regs[] = {
+	{ 0x818, 0x7ff }, { 0x810, 1 }, { 0x814, 0xffff },
+	{ SGX_CR_EVENT_HOST_ENABLE, SGX_CR_EVENT_STATUS_SW_EVENT },
+	{ SGX_CR_EVENT_HOST_ENABLE2, 0 },
+	{ 0x13c, 0 }, { 0xa7c, 0 }, { 0xa80, 0 }, { 0xa74, 0x208200 }, { 0x4e8, 0x100 },
+	{ 0xa78, 0x2c }, { 0xaac, 0x45 }, { 0xabc, 0xe }, { 0x800, 0x30 }, { 0x804, 0xffff },
+	{ 0xa00, 0x7c000 },
+};
+
+/*
+ * SGX530 (TI OMAP3630), from the GPL sgx530defs.h.
+ * It has no CLKGATECTL2, EVENT_KICK2 or BIF bank registers. The event PDS
+ * registers aren't in the GPL header: the SGX545's offsets, a guess here,
+ * checked by reading back before the first kick.
+ */
+static const struct pvrsgx_core_regs sgx530_regs = {
+	.core_id = 0x0010,
+	.core_revision = 0x0014,
+	/* 2D b0-1, ISP b4-5, TSP b8-9, TA b12-13, DPM b16-17, USE b20-21 */
+	.clkgatectl_units = 0x00333333,
+	.clkgatectl_dpm = GENMASK(17, 16),
+	.clkgatestatus = 0x0004,
+	.clkgatestatus_dpm = BIT(16),
+	.soft_reset_bif = BIT(0),
+	/* 2D b1, DPM b2, TA b3, USE b4, ISP b5, TSP b6 */
+	.soft_reset_cores = GENMASK(6, 1),
+	.use_code_base0 = SGX_CR_USE_CODE_BASE_0,
+	.pds_exec_base = SGX_CR_PDS_EXEC_BASE,
+	.bif_int_stat_fault_req = GENMASK(13, 0),
+	.bif_int_stat_fault_type = BIT(14),	/* read or write */
+	.bif_fault_addr = GENMASK(27, 12),
+	.use_code_base_dm_shift = 24,
+	/* 256-byte units: the EDM fetches at field << 8. */
+	.use_code_base_addr_shift = 8,
+	.event_pds_enable = SGX545_CR_EVENT_PDS_ENABLE,
+	.event_pds_enable_all = SGX545_CR_EVENT_PDS_ENABLE_ALL,
+	.event_pds_base = SGX545_CR_EVENT_PDS_BASE,
+	.event_pds_data0 = SGX545_CR_EVENT_PDS_DATA0,
+	.event_pds_data1 = SGX545_CR_EVENT_PDS_DATA1,
+};
+
+/* Init script part 2 for the SGX530: the host's interrupt, and one SGX545 write. */
+static const struct pvrsgx_core_init_reg sgx530_init_regs[] = {
+	{ SGX_CR_EVENT_HOST_ENABLE, SGX_CR_EVENT_STATUS_SW_EVENT },
+	{ SGX_CR_EVENT_HOST_ENABLE2, 0 },
+	/* Unnamed; without it the PDS fetches nothing. */
+	{ 0xa74, 0x208200 },
+};
+
+/*
+ * 256 MiB, every heap on a 4 MiB directory entry. From PDS pixel up, the
+ * bases of the GPL SGX530 sgxconfig.h: kernel code above both PDS
+ * heaps, within the 64 MiB the EDM reaches past PDS_EXEC_BASE. The texture
+ * heap, which that layout lacks, is the top of its general heap.
+ */
+static const struct pvrsgx_heap_layout sgx530_heaps = {
+	.general = { 0x01800000, 0x05000000 },
+	.texture = { 0x05000000, 0x08400000 },
+	.params_3d = { 0x08400000, 0x0c400000 },
+	.ta_data = { 0x0c400000, 0x0d400000 },
+	.sync = { 0x0d400000, 0x0d800000 },
+	.pds_pixel = { 0x0d800000, 0x0e000000 },
+	.pds_vertex = { 0x0e000000, 0x0e800000 },
+	.kernel_code = { 0x0ec00000, 0x0f000000 },
+	.kernel_data = { 0x0f000000, 0x0f400000 },
+	.usse_pixel = { 0x0f400000, 0x0f900000 },
+	.usse_vertex = { 0x0fc00000, 0x0fe00000 },
+	.pb_max_pages = 0x800,
+};
+
+static const struct pvrsgx_core_family families[] = {
+	{
+		.name = "SGX545",
+		.core = 545,
+		.core_id = 0x01150000,
+		.regs = &sgx545_regs,
+		.features = PVRSGX_FEATURE_MULTIPLE_MEM_CONTEXTS |
+			    PVRSGX_FEATURE_MULTI_EVENT_KICK |
+			    PVRSGX_FEATURE_PDS_DATA_INTERLEAVE_2DWORDS |
+			    PVRSGX_FEATURE_EDM_VERTEX_PDSADDR_FULL_RANGE,
+		.isa = PVRSGX_ISA_SGX545,
+		.num_dir_lists = 16,
+		.va_bits = 32,
+		.heaps = &sgx545_heaps,
+		.cmds = &pvrsgx_cmd_core_sgx545,
+		.init_regs = sgx545_init_regs,
+		.num_init_regs = ARRAY_SIZE(sgx545_init_regs),
+	},
+	{
+		/* One memory context, one kick (BRN_22934), 28-bit addresses. */
+		.name = "SGX530",
+		.core = 530,
+		/* Read on an OMAP3630 board. */
+		.core_id = 0x01120000,
+		.regs = &sgx530_regs,
+		.isa = PVRSGX_ISA_SGX545,
+		.num_dir_lists = 1,
+		.va_bits = 28,
+		.heaps = &sgx530_heaps,
+		.quirks = PVRSGX_QUIRK_BRN_28889,
+		.event_pds_guessed = true,
+		/* Its task's stores to EDMPROTECT pages fault. */
+		.data_no_edmprotect = true,
+		.cmds = &pvrsgx_cmd_core_sgx530,
+		.init_regs = sgx530_init_regs,
+		.num_init_regs = ARRAY_SIZE(sgx530_init_regs),
+	},
+};
+
+static const struct pvrsgx_core_info cores[] = {
+	{
+		.family = &families[0],
+		.name = "Intel CE5300 (SGX545 1.0.14)",
+		.core_rev = 0x0001000e,
+		.clock_hz = 400000000,
+		.fw_name = "powervr_sgx/sgx545_ukernel.bin",
+	},
+	{
+		/*
+		 * The same core one revision earlier: same features and instruction set, the same
+		 * microkernel. Its clock depends on the part (400 or 640 MHz);
+		 * the microkernel's timer only needs the order of magnitude.
+		 */
+		.family = &families[0],
+		.name = "Intel Cedarview (SGX545 1.0.13)",
+		.core_rev = 0x0001000d,
+		.clock_hz = 400000000,
+		.fw_name = "powervr_sgx/sgx545_ukernel.bin",
+	},
+	{
+		/*
+		 * Revision read on our OMAP3630 test board. The microkernel's
+		 * timer needs only the clock's order of magnitude.
+		 */
+		.family = &families[1],
+		.name = "TI OMAP3630 (SGX530 1.2.5)",
+		.core_rev = 0x00010205,
+		.clock_hz = 200000000,
+		.fw_name = "powervr_sgx/sgx530_ukernel.bin",
+	},
+};
+
+MODULE_FIRMWARE("powervr_sgx/sgx545_ukernel.bin");
+MODULE_FIRMWARE("powervr_sgx/sgx530_ukernel.bin");
+
+/*
+ * Without EDM_VERTEX_PDSADDR_FULL_RANGE, a heap the EDM runs PDS code from
+ * lies within 64 MiB above the PDS pixel heap (the GPL sgxconfig.h).
+ */
+static bool pds_reach(const struct pvrsgx_core_family *f, const struct pvrsgx_va_range *r)
+{
+	u64 exec = f->heaps->pds_pixel.base;
+
+	if (f->features & PVRSGX_FEATURE_EDM_VERTEX_PDSADDR_FULL_RANGE)
+		return true;
+	return r->base >= exec && r->end - exec <= SZ_64M;
+}
+
+static bool heaps_fit(const struct pvrsgx_core_family *f)
+{
+	const struct pvrsgx_heap_layout *h = f->heaps;
+	const struct pvrsgx_va_range *r[] = {
+		&h->params_3d, &h->ta_data, &h->sync, &h->pds_pixel, &h->kernel_code,
+		&h->pds_vertex, &h->kernel_data, &h->usse_pixel, &h->usse_vertex,
+		&h->general, &h->texture,
+	};
+
+	for (unsigned int i = 0; i < ARRAY_SIZE(r); i++)
+		if (r[i]->base >= r[i]->end || r[i]->end > BIT_ULL(f->va_bits))
+			return false;
+	return pds_reach(f, &h->kernel_code) && pds_reach(f, &h->pds_vertex);
+}
+
+/*
+ * Whether the driver can run the core, not only identify it: either a
+ * directory list per VM, or exactly one; a heap layout within its address
+ * space; the event PDS registers that enter the microkernel; and the second
+ * kick register exactly when the core has one.
+ */
+bool pvrsgx_core_family_usable(const struct pvrsgx_core_family *f)
+{
+	const struct pvrsgx_core_regs *cr = f->regs;
+	bool multi = f->features & PVRSGX_FEATURE_MULTIPLE_MEM_CONTEXTS;
+	bool kick2 = f->features & PVRSGX_FEATURE_MULTI_EVENT_KICK;
+
+	if (multi ? f->num_dir_lists < 2 || f->num_dir_lists > SGX_BIF_MAX_DIRLISTS :
+		    f->num_dir_lists != 1)
+		return false;
+	if ((f->va_bits != 28 && f->va_bits != 32) || !f->heaps || !heaps_fit(f) || !f->cmds)
+		return false;
+	return cr->event_pds_enable && cr->event_pds_base && kick2 == !!cr->event_kick2;
+}
+
+/*
+ * The core the bus says is there (545 for an SGX545), or NULL if the driver
+ * doesn't know where its registers are. Whether it can run it is
+ * pvrsgx_core_family_usable().
+ */
+const struct pvrsgx_core_family *pvrsgx_core_family_get(unsigned int core)
+{
+	for (unsigned int i = 0; i < ARRAY_SIZE(families); i++)
+		if (families[i].core == core)
+			return &families[i];
+	return NULL;
+}
+
+/* The entry for this part, or NULL if the driver doesn't know it. */
+const struct pvrsgx_core_info *pvrsgx_core_lookup(const struct pvrsgx_core_family *family,
+						  u32 core_id, u32 core_rev)
+{
+	if (core_id != family->core_id)
+		return NULL;
+	for (unsigned int i = 0; i < ARRAY_SIZE(cores); i++)
+		if (cores[i].family == family && cores[i].core_rev == core_rev)
+			return &cores[i];
+	return NULL;
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_core.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_core.h
new file mode 100644
index 000000000..58c1d4613
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_core.h
@@ -0,0 +1,196 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_CORE_H
+#define PVRSGX_CORE_H
+
+#include <linux/bits.h>
+#include <linux/types.h>
+
+struct pvrsgx_cmd_core;
+
+/*
+ * Core features, as the SGX_FEATURE_* names of the GPL sgxfeaturedefs.h.
+ * Only those the driver's design depends on are listed.
+ */
+/* One page directory per client: BIF directory lists beyond list 0. */
+#define PVRSGX_FEATURE_MULTIPLE_MEM_CONTEXTS	BIT(0)
+/* EUR_CR_EVENT_KICK1..3 next to EUR_CR_EVENT_KICK. */
+#define PVRSGX_FEATURE_MULTI_EVENT_KICK		BIT(1)
+/* The PDS data store interleaves DS0 and DS1 two words at a time. */
+#define PVRSGX_FEATURE_PDS_DATA_INTERLEAVE_2DWORDS	BIT(2)
+/*
+ * The EDM reaches PDS code anywhere. Without it, the kernel code and PDS
+ * vertex heaps lie within 64 MiB above the PDS pixel heap (PDS_EXEC_BASE):
+ * the GPL sgxconfig.h refuses any other layout.
+ */
+#define PVRSGX_FEATURE_EDM_VERTEX_PDSADDR_FULL_RANGE	BIT(3)
+
+/* Errata the kernel works around (the GPL headers' FIX_HW_BRN_*). */
+/* A BIF directory or table invalidate and a data invalidate never in one command. */
+#define PVRSGX_QUIRK_BRN_28889			BIT(0)
+
+/* The USSE instruction set, for the shader validator's tables. */
+enum pvrsgx_isa {
+	PVRSGX_ISA_SGX545,
+};
+
+/* A device-virtual range, [base, end). */
+struct pvrsgx_va_range {
+	u64 base;
+	u64 end;
+};
+
+/*
+ * struct pvrsgx_heap_layout - Where every heap sits
+ *
+ * Device-wide, the same in every VM. The kernel heaps (kernel code, kernel
+ * data, sync) are whole 4 MiB directory entries.
+ */
+struct pvrsgx_heap_layout {
+	struct pvrsgx_va_range params_3d;
+	struct pvrsgx_va_range ta_data;
+	struct pvrsgx_va_range sync;
+	struct pvrsgx_va_range pds_pixel;
+	struct pvrsgx_va_range kernel_code;
+	struct pvrsgx_va_range pds_vertex;
+	struct pvrsgx_va_range kernel_data;
+	struct pvrsgx_va_range usse_pixel;
+	struct pvrsgx_va_range usse_vertex;
+	struct pvrsgx_va_range general;
+	struct pvrsgx_va_range texture;
+	/* The largest PB, in pages, the 3D parameters heap holds. */
+	u32 pb_max_pages;
+};
+
+/*
+ * struct pvrsgx_core_regs - Registers that move or change between SGX5xx cores
+ *
+ * The other registers in pvrsgx_regs.h sit at the same offsets with the same
+ * fields on SGX535, 540, 544 and 545 (the GPL sgx5xxdefs.h). A register
+ * a core lacks is 0.
+ */
+struct pvrsgx_core_regs {
+	u32 core_id;
+	u32 core_revision;
+	u32 clkgatectl_units;		/* every gating field of CLKGATECTL */
+	u32 clkgatectl_dpm;		/* CLKGATECTL's DPM field */
+	u32 clkgatectl2;
+	u32 clkgatectl2_units;
+	u32 clkgatestatus;
+	u32 clkgatestatus_dpm;
+	u32 soft_reset_bif;
+	u32 soft_reset_cores;		/* every unit's reset but the BIF's */
+	u32 event_kick2;
+	u32 event_kick2_now;
+	u32 use_code_base0;
+	u32 pds_exec_base;
+	u32 bif_int_stat_fault_req;
+	u32 bif_int_stat_fault_type;
+	u32 bif_fault_addr;		/* BIF_FAULT's address field */
+	u32 bif_bank_set;
+	u32 bif_bank0;
+	u32 use_code_base_dm_shift;	/* USE_CODE_BASE's data-master field */
+	u32 use_code_base_addr_shift;	/* USE_CODE_BASE's address unit, log2 bytes */
+	u32 cr_b30;			/* unnamed, 0x100 at start on the SGX545 */
+	/* Not in the GPL header (pvrsgx_hw.h). */
+	u32 event_pds_enable;
+	u32 event_pds_enable_all;
+	u32 event_pds_base;
+	u32 event_pds_data0;
+	u32 event_pds_data1;
+	u32 edm_task_reg0;
+	u32 edm_task_reg1;
+};
+
+/* One write of a core's init script. */
+struct pvrsgx_core_init_reg {
+	u16 reg;
+	u32 val;
+};
+
+/**
+ * struct pvrsgx_core_family - What one SGX5xx core design has in common
+ *
+ * Every revision and part of the core shares these. The bus says which core
+ * to expect, since even SGX_CR_CORE_ID moves between cores.
+ */
+struct pvrsgx_core_family {
+	/** @name: The core, as "SGX545". */
+	const char *name;
+	/** @core: Its number, as the bus glue names it: 545 for SGX545. */
+	unsigned int core;
+	/** @core_id: Expected SGX_CR_CORE_ID. */
+	u32 core_id;
+	/** @regs: Where its registers and fields are. */
+	const struct pvrsgx_core_regs *regs;
+	/** @features: PVRSGX_FEATURE_*. */
+	u32 features;
+	/** @isa: Its USSE instruction set. */
+	enum pvrsgx_isa isa;
+	/** @num_dir_lists: BIF directory lists (SGX_FEATURE_BIF_NUM_DIRLISTS). */
+	u32 num_dir_lists;
+	/** @va_bits: Device-virtual address bits (SGX_FEATURE_ADDRESS_SPACE_SIZE). */
+	u32 va_bits;
+	/** @heaps: The device-virtual layout, within @va_bits. */
+	const struct pvrsgx_heap_layout *heaps;
+	/** @quirks: PVRSGX_QUIRK_*. */
+	u32 quirks;
+	/**
+	 * @event_pds_guessed: The event PDS registers are a guess, not yet
+	 * confirmed on this core: refuse to kick if the program address
+	 * doesn't read back.
+	 */
+	bool event_pds_guessed;
+	/**
+	 * @data_no_edmprotect: Map the kernel data arena without EDMPROTECT. A
+	 * bring-up stopgap for the SGX530, whose microkernel task's stores to an
+	 * EDMPROTECT page fault. It runs only with allow_unisolated, which
+	 * already gives up client isolation on such a core.
+	 */
+	bool data_no_edmprotect;
+	/** @cmds: What its client commands are checked against (pvrsgx_cmd_defs.c). */
+	const struct pvrsgx_cmd_core *cmds;
+	/** @init_regs: Init script part 2's register writes, in order. */
+	const struct pvrsgx_core_init_reg *init_regs;
+	/** @num_init_regs: Entries in @init_regs. */
+	unsigned int num_init_regs;
+};
+
+/**
+ * struct pvrsgx_core_info - One SGX5xx part this driver supports
+ *
+ * A new part of a known core is a new entry; a new core is a new
+ * &struct pvrsgx_core_family as well. The microkernel host interface and the
+ * client heaps are the same across the family.
+ */
+struct pvrsgx_core_info {
+	/** @family: The core design. */
+	const struct pvrsgx_core_family *family;
+	/** @name: What the part is called in the log. */
+	const char *name;
+	/** @core_rev: Expected SGX_CR_CORE_REVISION. */
+	u32 core_rev;
+	/** @clock_hz: Core clock, for the microkernel's timer. */
+	u32 clock_hz;
+	/** @fw_name: The open microkernel under /lib/firmware. */
+	const char *fw_name;
+};
+
+/*
+ * One memory context: the microkernel, TA, 3D and 2D all translate through
+ * directory list 0, so the firmware heaps are in every VM's directory.
+ */
+static inline bool pvrsgx_core_single_context(const struct pvrsgx_core_family *f)
+{
+	return !(f->features & PVRSGX_FEATURE_MULTIPLE_MEM_CONTEXTS);
+}
+
+extern bool pvrsgx_allow_unisolated;
+
+const struct pvrsgx_core_family *pvrsgx_core_family_get(unsigned int core);
+bool pvrsgx_core_family_usable(const struct pvrsgx_core_family *f);
+const struct pvrsgx_core_info *pvrsgx_core_lookup(const struct pvrsgx_core_family *family,
+						  u32 core_id, u32 core_rev);
+
+#endif /* PVRSGX_CORE_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_device.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_device.c
new file mode 100644
index 000000000..9328e8e06
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_device.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+#include <drm/drm_managed.h>
+#include <drm/drm_print.h>
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_GMA500)
+#include <drm/gma500_sgx.h>
+#endif
+
+#include <linux/bitfield.h>
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/jiffies.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+
+#include "pvrsgx_ccb.h"
+#include "pvrsgx_core.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_trace.h"
+#include "pvrsgx_kobj.h"
+#include "pvrsgx_mmu.h"
+#include "pvrsgx_render.h"
+
+int pvrsgx_poll(struct pvrsgx_device *sdev, const u32 *addr, u32 mask, u32 val,
+		unsigned int timeout_us)
+{
+	ktime_t end = ktime_add_us(ktime_get(), timeout_us);
+
+	for (;;) {
+		if ((READ_ONCE(*addr) & mask) == val)
+			return 0;
+		if (ktime_after(ktime_get(), end))
+			return (READ_ONCE(*addr) & mask) == val ? 0 : -ETIMEDOUT;
+		usleep_range(20, 50);
+	}
+}
+
+/*
+ * Kernel-owned memory at a fixed device address, mapped in the firmware's
+ * directory. Pages below 4 GiB, write-combined in every CPU mapping. gfp adds
+ * to GFP_KERNEL: __GFP_ACCOUNT for what a client's ioctl allocates.
+ */
+int pvrsgx_kmem_alloc(struct pvrsgx_device *sdev, struct pvrsgx_kmem *km,
+		      u32 dev_addr, u32 size, u32 pte_flags, gfp_t gfp)
+{
+	unsigned int i;
+	int err = -ENOMEM;
+
+	km->dev_addr = dev_addr;
+	km->size = PAGE_ALIGN(size);
+	km->npages = km->size >> PAGE_SHIFT;
+	km->pages = kvcalloc(km->npages, sizeof(*km->pages), GFP_KERNEL | gfp);
+	if (!km->pages)
+		return -ENOMEM;
+	for (i = 0; i < km->npages; i++) {
+		km->pages[i] = alloc_page(GFP_KERNEL | __GFP_ZERO | __GFP_DMA32 | gfp);
+		if (!km->pages[i])
+			goto err_free;
+	}
+	if (pvrsgx_pages_wc(sdev->base.dev, km->pages, km->npages))
+		goto err_free;
+	km->cpu = vmap(km->pages, km->npages, VM_MAP, pgprot_writecombine(PAGE_KERNEL));
+	if (!km->cpu)
+		goto err_wb;
+	if (!km->mmu)
+		km->mmu = sdev->kernel_mmu;
+	err = pvrsgx_mmu_map_pages(km->mmu, dev_addr, km->pages, km->npages, pte_flags);
+	if (err)
+		goto err_unmap;
+	return 0;
+
+err_unmap:
+	vunmap(km->cpu);
+err_wb:
+	pvrsgx_pages_wb(km->pages, km->npages);
+err_free:
+	while (i--)
+		__free_page(km->pages[i]);
+	kvfree(km->pages);
+	km->pages = NULL;
+	return err;
+}
+
+void pvrsgx_kmem_free(struct pvrsgx_device *sdev, struct pvrsgx_kmem *km)
+{
+	if (!km->pages)
+		return;
+	pvrsgx_mmu_unmap(km->mmu, km->dev_addr, km->size);
+	pvrsgx_kmem_release(km);
+}
+
+/* The memory of kmem the caller has already unmapped from the GPU. */
+void pvrsgx_kmem_release(struct pvrsgx_kmem *km)
+{
+	if (!km->pages)
+		return;
+	vunmap(km->cpu);
+	pvrsgx_pages_wb(km->pages, km->npages);
+	for (unsigned int i = 0; i < km->npages; i++)
+		__free_page(km->pages[i]);
+	kvfree(km->pages);
+	km->pages = NULL;
+}
+
+/* A few core clocks for a register write to take effect. */
+static void reset_sleep(struct pvrsgx_device *sdev)
+{
+	sgx_rd(sdev, SGX_CR_SOFT_RESET);
+	udelay(1);
+}
+
+static void soft_reset(struct pvrsgx_device *sdev, bool bif)
+{
+	sgx_wr(sdev, SGX_CR_SOFT_RESET,
+	       sdev->cr->soft_reset_cores | (bif ? sdev->cr->soft_reset_bif : 0));
+}
+
+/* BIF_CTRL is 0 whenever the microkernel runs: its invalidates pulse a bit, then write 0. */
+static void inval_dc(struct pvrsgx_device *sdev)
+{
+	sgx_wr(sdev, SGX_CR_BIF_CTRL, SGX_CR_BIF_CTRL_INVALDC);
+	sgx_wr(sdev, SGX_CR_BIF_CTRL, 0);
+	reset_sleep(sdev);
+}
+
+/*
+ * The reset sequence for a single-core SGX5xx: reset everything, clear the directory
+ * lists, take the cores out of reset with list 0 on a scratch directory that
+ * absorbs faults left over from before, then give the microkernel its own
+ * directory on the last list (list 15 on the SGX545).
+ */
+void pvrsgx_reset(struct pvrsgx_device *sdev, bool recovery)
+{
+	soft_reset(sdev, true);
+	reset_sleep(sdev);
+
+	sgx_wr(sdev, SGX_CR_BIF_CTRL, 0);
+	sgx_wr_opt(sdev, sdev->cr->bif_bank_set, 0);
+	sgx_wr_opt(sdev, sdev->cr->bif_bank0, 0);
+	for (unsigned int i = 0; i < sdev->family->num_dir_lists; i++)
+		sgx_wr(sdev, sgx_cr_bif_dir_list_base(i), 0);
+
+	if (recovery) {
+		sgx_wr(sdev, sgx_cr_bif_dir_list_base(0), pvrsgx_mmu_pd_addr(sdev->reset_mmu));
+		reset_sleep(sdev);
+		soft_reset(sdev, false);
+		reset_sleep(sdev);
+		inval_dc(sdev);
+
+		/* Let each faulting request complete on a scratch page. */
+		for (int n = 0; n < 32; n++) {
+			u32 stat = sgx_rd(sdev, SGX_CR_BIF_INT_STAT), addr;
+
+			if (!(stat & sdev->cr->bif_int_stat_fault_req))
+				break;
+			addr = sgx_rd(sdev, SGX_CR_BIF_FAULT) & sdev->cr->bif_fault_addr;
+			drm_warn(&sdev->base, "reset: page fault 0x%x at 0x%08x\n", stat, addr);
+			soft_reset(sdev, true);
+			pvrsgx_mmu_map_pages(sdev->reset_mmu, addr, &sdev->reset_page, 1, 0);
+			sgx_wr(sdev, SGX_CR_EVENT_HOST_CLEAR, sgx_rd(sdev, SGX_CR_EVENT_STATUS));
+			sgx_wr(sdev, SGX_CR_EVENT_HOST_CLEAR2, sgx_rd(sdev, SGX_CR_EVENT_STATUS2));
+			/* The reset page's PTE is in memory before the cores leave reset. */
+			wmb();
+			reset_sleep(sdev);
+			soft_reset(sdev, false);
+			reset_sleep(sdev);
+			inval_dc(sdev);
+			pvrsgx_mmu_unmap(sdev->reset_mmu, addr, SGX_MMU_PAGE_SIZE);
+		}
+	} else {
+		soft_reset(sdev, false);
+		reset_sleep(sdev);
+	}
+
+	sgx_wr_opt(sdev, sdev->cr->bif_bank0,
+		   FIELD_PREP(SGX_CR_BIF_BANK0_INDEX_EDM, pvrsgx_dirlist_edm(sdev)));
+	/* With one directory list the EDM's is list 0, set below. */
+	if (pvrsgx_dirlist_edm(sdev))
+		sgx_wr(sdev, sgx_cr_bif_dir_list_base(pvrsgx_dirlist_edm(sdev)),
+		       pvrsgx_mmu_pd_addr(sdev->kernel_mmu));
+	sgx_wr(sdev, sgx_cr_bif_dir_list_base(0),
+	       pvrsgx_mmu_pd_addr(pvrsgx_idle_mmu(sdev)));
+	inval_dc(sdev);
+
+	sgx_wr(sdev, SGX_CR_SOFT_RESET, 0);
+	reset_sleep(sdev);
+}
+
+/* The microkernel raises SW_EVENT when it wants the host. */
+static irqreturn_t pvrsgx_irq(int irq, void *arg)
+{
+	struct pvrsgx_device *sdev = arg;
+	u32 status;
+
+	if (!READ_ONCE(sdev->powered))
+		return IRQ_NONE;
+	status = sgx_rd(sdev, SGX_CR_EVENT_STATUS) & sgx_rd(sdev, SGX_CR_EVENT_HOST_ENABLE);
+	if (!(status & SGX_CR_EVENT_STATUS_SW_EVENT))
+		return IRQ_NONE;
+	trace_pvrsgx_irq(status);
+	sgx_wr(sdev, SGX_CR_EVENT_HOST_CLEAR,
+	       SGX_CR_EVENT_HOST_CLEAR_SW_EVENT | SGX_CR_EVENT_HOST_CLEAR_MASTER_INTERRUPT);
+	atomic_inc(&sdev->irq_count);
+	/*
+	 * The microkernel may interrupt at TA and 3D completion too: wake the
+	 * thread only for a finished job or a lockup report.
+	 */
+	if (!sdev->fw_running || pvrsgx_job_oldest_done(sdev) ||
+	    (pvrsgx_fw_hostctl(sdev, HOSTCTL_INTERRUPT_FLAGS) & HOSTCTL_INTERRUPT_HWR))
+		return IRQ_WAKE_THREAD;
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t pvrsgx_irq_thread(int irq, void *arg)
+{
+	struct pvrsgx_device *sdev = arg;
+	u32 flags, clear;
+
+	if (!sdev->fw_running)
+		return IRQ_HANDLED;
+
+	flags = pvrsgx_fw_hostctl(sdev, HOSTCTL_INTERRUPT_FLAGS);
+	clear = pvrsgx_fw_hostctl(sdev, HOSTCTL_INTERRUPT_CLEAR);
+	if ((flags & HOSTCTL_INTERRUPT_HWR) && !(clear & HOSTCTL_INTERRUPT_HWR))
+		drm_err_ratelimited(&sdev->base, "microkernel reports a lockup (%u)\n",
+				    pvrsgx_fw_hostctl(sdev, HOSTCTL_UK_LOCKUPS));
+	/*
+	 * The microkernel's idle notice: acknowledge it in the interrupt clear word.
+	 * Powering down is runtime PM's business, so nothing else happens here.
+	 */
+	if ((flags & ~clear) & HOSTCTL_INTERRUPT_ACTIVE_POWER)
+		pvrsgx_fw_hostctl_set(sdev, HOSTCTL_INTERRUPT_CLEAR,
+				      clear | HOSTCTL_INTERRUPT_ACTIVE_POWER);
+
+	pvrsgx_job_process_done(sdev);
+	return IRQ_HANDLED;
+}
+
+/*
+ * Which core this is (pvrsgx_core.c). Anything else is refused before the
+ * driver writes a register: its errata, clocks and microkernel are unknown.
+ */
+static int identify(struct pvrsgx_device *sdev)
+{
+	if (!sdev->family) {
+		drm_err(&sdev->base, "not an SGX core this driver knows; not touching it\n");
+		return -ENODEV;
+	}
+	sdev->cr = sdev->family->regs;
+
+	/* Safe on a core in any state. */
+	sdev->core_id = sgx_rd(sdev, sdev->cr->core_id);
+	sdev->core_rev = sgx_rd(sdev, sdev->cr->core_revision);
+
+	if (!pvrsgx_core_family_usable(sdev->family)) {
+		drm_info(&sdev->base,
+			 "%s core id 0x%08x revision 0x%08x: this driver can't run an %s yet; not touching it further\n",
+			 sdev->family->name, sdev->core_id, sdev->core_rev, sdev->family->name);
+		return -ENODEV;
+	}
+	if (pvrsgx_core_single_context(sdev->family)) {
+		if (!pvrsgx_allow_unisolated) {
+			drm_err(&sdev->base,
+				"%s has one memory context: every client could reach the firmware and the other clients' commands. Not running it; powervr_sgx.allow_unisolated=1 runs it anyway.\n",
+				sdev->family->name);
+			return -ENODEV;
+		}
+		drm_warn(&sdev->base,
+			 "%s has one memory context: clients are not isolated from the firmware or from each other (allow_unisolated=1)\n",
+			 sdev->family->name);
+	}
+
+	sdev->core = pvrsgx_core_lookup(sdev->family, sdev->core_id, sdev->core_rev);
+	if (!sdev->core) {
+		drm_err(&sdev->base,
+			"%s core id 0x%08x revision 0x%08x is not one this driver knows; not touching it. Please report these two values.\n",
+			sdev->family->name, sdev->core_id, sdev->core_rev);
+		return -ENODEV;
+	}
+	drm_info(&sdev->base, "SGX core id 0x%08x revision 0x%08x: %s\n",
+		 sdev->core_id, sdev->core_rev, sdev->core->name);
+	return 0;
+}
+
+/*
+ * On a single-context core the microkernel runs on the job's directory, so
+ * every VM maps the firmware heaps. They get their page tables now, so the
+ * directory entries every VM copies never change.
+ */
+static int share_fw_heaps_init(struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_heap_layout *h = pvrsgx_heaps(sdev);
+	const struct pvrsgx_va_range *r[] = { &h->kernel_code, &h->kernel_data, &h->sync };
+	int err = 0;
+
+	if (!pvrsgx_core_single_context(sdev->family))
+		return 0;
+	for (unsigned int i = 0; i < ARRAY_SIZE(r) && !err; i++)
+		err = pvrsgx_mmu_prealloc(sdev->kernel_mmu, r[i]->base, r[i]->end);
+	return err;
+}
+
+/* Point a VM's directory at the firmware heaps' page tables, on a single-context core. */
+void pvrsgx_share_fw_heaps(struct pvrsgx_device *sdev, struct pvrsgx_mmu *mmu)
+{
+	const struct pvrsgx_heap_layout *h = pvrsgx_heaps(sdev);
+
+	if (!pvrsgx_core_single_context(sdev->family))
+		return;
+	pvrsgx_mmu_share(mmu, sdev->kernel_mmu, h->kernel_code.base, h->kernel_code.end);
+	pvrsgx_mmu_share(mmu, sdev->kernel_mmu, h->kernel_data.base, h->kernel_data.end);
+	pvrsgx_mmu_share(mmu, sdev->kernel_mmu, h->sync.base, h->sync.end);
+}
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_GMA500)
+/* Behind gma500 the hard handler runs from gma500's; its thread is a work item. */
+static irqreturn_t pvrsgx_gma_irq(void *data)
+{
+	struct pvrsgx_device *sdev = data;
+	irqreturn_t ret = pvrsgx_irq(sdev->irq, sdev);
+
+	if (ret == IRQ_WAKE_THREAD) {
+		queue_work(system_highpri_wq, &sdev->irq_work);
+		return IRQ_HANDLED;
+	}
+	return ret;
+}
+
+static void pvrsgx_gma_irq_work(struct work_struct *work)
+{
+	struct pvrsgx_device *sdev = container_of(work, struct pvrsgx_device, irq_work);
+
+	pvrsgx_irq_thread(sdev->irq, sdev);
+}
+#endif
+
+static int irq_setup(struct pvrsgx_device *sdev)
+{
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_GMA500)
+	if (sdev->gma_sgx) {
+		INIT_WORK(&sdev->irq_work, pvrsgx_gma_irq_work);
+		return sdev->gma_sgx->set_irq_handler(sdev->gma_sgx, pvrsgx_gma_irq, sdev);
+	}
+#endif
+	return request_threaded_irq(sdev->irq, pvrsgx_irq, pvrsgx_irq_thread, IRQF_SHARED,
+				    "powervr_sgx", sdev);
+}
+
+static void irq_teardown(struct pvrsgx_device *sdev)
+{
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_GMA500)
+	if (sdev->gma_sgx) {
+		sdev->gma_sgx->set_irq_handler(sdev->gma_sgx, NULL, NULL);
+		cancel_work_sync(&sdev->irq_work);
+		return;
+	}
+#endif
+	free_irq(sdev->irq, sdev);
+}
+
+/*
+ * The software state, at the DRM device's release: after unbind, and after the
+ * last file is closed, so nothing a file can reach goes before the file does.
+ */
+static void pvrsgx_device_release(struct drm_device *drm, void *arg)
+{
+	struct pvrsgx_device *sdev = arg;
+
+	pvrsgx_job_fini(sdev);
+	pvrsgx_ccb_inval_reap(sdev, true);
+	pvrsgx_pb_put(sdev->pb);
+	mutex_destroy(&sdev->pb_lock);
+	pvrsgx_kobj_fini(sdev);
+	pvrsgx_fw_fini(sdev);
+	__free_page(sdev->reset_page);
+	/* The reset directory still names the scratch page's tables, never the page. */
+	pvrsgx_mmu_destroy(sdev->reset_mmu);
+	pvrsgx_mmu_destroy(sdev->idle_mmu);
+	pvrsgx_mmu_destroy(sdev->kernel_mmu);
+}
+
+int pvrsgx_device_init(struct pvrsgx_device *sdev)
+{
+	int err;
+
+	mutex_init(&sdev->ccb_lock);
+	mutex_init(&sdev->done_lock);
+	spin_lock_init(&sdev->ccb_spin);
+	spin_lock_init(&sdev->inval_lock);
+	INIT_LIST_HEAD(&sdev->inval_waits);
+
+	sdev->kernel_mmu = pvrsgx_mmu_create(sdev);
+	sdev->idle_mmu = pvrsgx_mmu_create(sdev);
+	sdev->reset_mmu = pvrsgx_mmu_create(sdev);
+	sdev->reset_page = alloc_page(GFP_KERNEL | __GFP_ZERO | __GFP_DMA32);
+	if (IS_ERR(sdev->kernel_mmu) || IS_ERR(sdev->idle_mmu) ||
+	    IS_ERR(sdev->reset_mmu) || !sdev->reset_page) {
+		err = -ENOMEM;
+		goto err_mmu;
+	}
+
+	err = identify(sdev);
+	if (err)
+		goto err_mmu;
+	err = share_fw_heaps_init(sdev);
+	if (err)
+		goto err_mmu;
+
+	pvrsgx_kobj_init(sdev);
+	mutex_init(&sdev->pb_lock);
+	err = pvrsgx_fw_init(sdev);
+	if (err)
+		goto err_fw;
+
+	err = pvrsgx_job_init(sdev);
+	if (err)
+		goto err_fw;
+
+	/* From here the release frees the software state, on success or failure. */
+	err = drmm_add_action_or_reset(&sdev->base, pvrsgx_device_release, sdev);
+	if (err)
+		return err;
+
+	err = irq_setup(sdev);
+	if (err)
+		return err;
+
+	err = pvrsgx_fw_start(sdev);
+	if (err) {
+		pvrsgx_fw_stop(sdev);
+		irq_teardown(sdev);
+	}
+	return err;
+
+err_fw:
+	pvrsgx_fw_fini(sdev);
+	pvrsgx_kobj_fini(sdev);
+err_mmu:
+	if (sdev->reset_page)
+		__free_page(sdev->reset_page);
+	pvrsgx_mmu_destroy(sdev->reset_mmu);
+	pvrsgx_mmu_destroy(sdev->idle_mmu);
+	pvrsgx_mmu_destroy(sdev->kernel_mmu);
+	return err;
+}
+
+/*
+ * The hardware, at unbind or when registration fails: no job reaches the GPU
+ * again, the firmware stops, the interrupt goes. Open files keep working
+ * against the software state until they close.
+ */
+void pvrsgx_device_unplug(struct pvrsgx_device *sdev)
+{
+	pvrsgx_job_unplug(sdev);
+	irq_teardown(sdev);
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_device.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_device.h
new file mode 100644
index 000000000..0342b45d4
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_device.h
@@ -0,0 +1,354 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_DEVICE_H
+#define PVRSGX_DEVICE_H
+
+#include <drm/drm_device.h>
+#include <drm/drm_file.h>
+#include <drm/drm_mm.h>
+#include <drm/gpu_scheduler.h>
+
+#include <linux/device.h>
+#include <linux/idr.h>
+#include <linux/io.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/workqueue.h>
+#include <linux/xarray.h>
+
+#include "pvrsgx_cache.h"
+#include "pvrsgx_core.h"
+#include "pvrsgx_hw.h"
+
+/* Device-wide ranges for kernel-owned objects. */
+enum pvrsgx_krange {
+	PVRSGX_KRANGE_KDATA,	/* firmware only */
+	PVRSGX_KRANGE_KCODE,	/* firmware only */
+	PVRSGX_KRANGE_SYNC,	/* firmware only */
+	PVRSGX_KRANGE_3DPARAMS,	/* firmware and the owner's VM */
+	PVRSGX_KRANGE_TADATA,	/* firmware and the owner's VM */
+	PVRSGX_KRANGE_COUNT
+};
+
+struct firmware;
+struct pvrsgx_core_info;
+struct gma500_sgx_device;
+struct pci_dev;
+struct pvrsgx_mmu;
+struct pvrsgx_pb;
+struct pvrsgx_vm;
+
+/**
+ * struct pvrsgx_kmem - Kernel-owned memory at a fixed device address
+ *
+ * Mapped in the firmware's page directory only, never in a client's. The CPU
+ * mapping is write-combined.
+ */
+struct pvrsgx_kmem {
+	/** @mmu: Directory it is mapped in; NULL (the default) is the firmware's. */
+	struct pvrsgx_mmu *mmu;
+	/** @dev_addr: Device address. */
+	u32 dev_addr;
+	/** @size: Size in bytes, a whole number of pages. */
+	u32 size;
+	/** @npages: Number of pages. */
+	unsigned int npages;
+	/** @pages: The pages, below 4 GiB. */
+	struct page **pages;
+	/** @cpu: Write-combined kernel mapping. */
+	void *cpu;
+};
+
+/* Firmware-visible buffers, in allocation order. */
+enum pvrsgx_fw_buf {
+	PVRSGX_FW_CCB,		/* kernel CCB: 256 commands */
+	PVRSGX_FW_CCBCTL,	/* its write and read offsets */
+	PVRSGX_FW_KICKER,	/* EUR_CR_EVENT_KICKER target */
+	PVRSGX_FW_TA3DCTL,	/* the microkernel's control block */
+	PVRSGX_FW_HOSTCTL,	/* host control block */
+	PVRSGX_FW_MISC,		/* misc info the microkernel may fill */
+	PVRSGX_FW_UKERNEL,	/* USSE microkernel */
+	PVRSGX_FW_PDS0,		/* PDS programs that enter it */
+	PVRSGX_FW_PDS1,
+	PVRSGX_FW_PDS2,
+	PVRSGX_FW_PDS3,
+	PVRSGX_FW_PDS4,
+	PVRSGX_FW_NBUF
+};
+
+/**
+ * struct pvrsgx_fw - The running microkernel and what it reads
+ */
+struct pvrsgx_fw {
+	/** @data: Kernel data heap: control blocks, CCB. */
+	struct pvrsgx_kmem data;
+	/** @code: Kernel code heap: microkernel and its PDS programs. */
+	struct pvrsgx_kmem code;
+	/** @pdtable: Single-context cores: the directory of each VM by slot, GPU read-only. */
+	struct pvrsgx_kmem pdtable;
+	/** @pd_slots: The @pdtable slots VMs hold. */
+	struct ida pd_slots;
+	/** @addr: Device address of each buffer. */
+	u32 addr[PVRSGX_FW_NBUF];
+	/** @cpu: CPU address of each buffer. */
+	void *cpu[PVRSGX_FW_NBUF];
+	/** @cache_control: Invalidates to request with the next command. */
+	u32 cache_control;
+	/** @interface: DRM_POWERVR_SGX_FW_IF_*. */
+	u32 interface;
+	/** @name: Firmware file the image came from. */
+	const char *name;
+	/** @image: The microkernel as loaded, for restarts. */
+	u8 *image;
+	/** @image_size: Bytes of USSE code in @image. */
+	u32 image_size;
+	/** @features: PVRSGX_UKFW_* the microkernel's file header declares. */
+	u32 features;
+	/** @entries: The microkernel's entry table (PDS program, offset), or NULL. */
+	u32 (*entries)[2];
+	/** @n_entries: Entries in @entries. */
+	u32 n_entries;
+	/** @patches: The microkernel's patch table (offset, value), or NULL. */
+	u32 (*patches)[2];
+	/** @n_patches: Entries in @patches. */
+	u32 n_patches;
+};
+
+/**
+ * struct pvrsgx_device - The SGX545
+ */
+struct pvrsgx_device {
+	/** @base: The DRM device. */
+	struct drm_device base;
+	/** @pdev: The SGX PCI function (CE5300), or NULL behind gma500 (Cedarview). */
+	struct pci_dev *pdev;
+	/** @gma_sgx: The gma500 auxiliary device behind which the SGX sits, or NULL. */
+	struct gma500_sgx_device *gma_sgx;
+	/** @irq_work: The interrupt's bottom half behind gma500, which owns the line. */
+	struct work_struct irq_work;
+
+	/** @regs: The core registers (BAR 0). */
+	void __iomem *regs;
+	/** @irq: Its interrupt. */
+	int irq;
+
+	/** @family: The core the bus says is there, set before the probe reads a register. */
+	const struct pvrsgx_core_family *family;
+	/** @cr: Its registers that move between cores (@family's). */
+	const struct pvrsgx_core_regs *cr;
+	/** @core: What the driver knows about this part. */
+	const struct pvrsgx_core_info *core;
+	/** @core_id: SGX_CR_CORE_ID. */
+	u32 core_id;
+	/** @core_rev: SGX_CR_CORE_REVISION. */
+	u32 core_rev;
+
+	/** @kernel_mmu: Page directory the microkernel translates through. */
+	struct pvrsgx_mmu *kernel_mmu;
+	/** @idle_mmu: Empty directory for list 0 while no job runs (multi-context cores). */
+	struct pvrsgx_mmu *idle_mmu;
+	/** @reset_mmu: Scratch directory that absorbs faults during reset. */
+	struct pvrsgx_mmu *reset_mmu;
+	/** @reset_page: The page every address of @reset_mmu maps. */
+	struct page *reset_page;
+
+	/** @fw: The microkernel and its buffers. */
+	struct pvrsgx_fw fw;
+	/** @fw_running: The microkernel has started and takes commands. */
+	bool fw_running;
+	/**
+	 * @powered: The SGX is clocked and its registers may be read. Off between
+	 * a stop and the next start: a read of an unclocked CE5300 block can hang
+	 * the bus, and the interrupt line is shared.
+	 */
+	bool powered;
+	/** @fw_started: The microkernel has run since load; later starts are restarts. */
+	bool fw_started;
+	/** @fw_gen: Restarts (resets and runtime resumes) since load. */
+	u32 fw_gen;
+	/** @lost: Firmware failed to restart; every job fails with -EIO. */
+	bool lost;
+	/** @reset_lock: Serializes a GPU reset with unbind. */
+	struct mutex reset_lock;
+	/** @faulted: The BIF reported a page fault; the running job ends with -EFAULT. */
+	bool faulted;
+
+	/** @ccb_lock: Serialises kernel CCB writers. */
+	struct mutex ccb_lock;
+	/** @ccb_spin: Keeps @ccb_issued and the write offset in step for readers. */
+	spinlock_t ccb_spin;
+	/** @ccb_issued: Commands written to the kernel CCB since load, across restarts. */
+	u64 ccb_issued;
+	/** @inval_lock: Protects @inval_waits. */
+	spinlock_t inval_lock;
+	/** @inval_waits: &struct pvrsgx_inval_wait for invalidates not yet taken. */
+	struct list_head inval_waits;
+
+	/** @sched: The one scheduler; one job on the GPU at a time. */
+	struct drm_gpu_scheduler sched;
+	/** @fence_context: Context of the hardware fences. */
+	u64 fence_context;
+	/** @done_lock: Keeps completions in order while imports are flushed. */
+	struct mutex done_lock;
+	/** @fence_lock: Protects @fence_seqno and @active. */
+	spinlock_t fence_lock;
+	/** @fence_seqno: Last hardware fence seqno. */
+	u32 fence_seqno;
+	/** @active: Jobs handed to the firmware, oldest first. */
+	struct list_head active;
+	/** @last_fence: Hardware fence of the job last sent to the firmware. */
+	struct dma_fence *last_fence;
+	/**
+	 * @wq: Runs @poll_work, which signals fences: the queue has a rescuer
+	 * and never waits on memory reclaim.
+	 */
+	struct workqueue_struct *wq;
+	/**
+	 * @free_wq: Runs @reap_work and the jobs' final puts, which take
+	 * reservations and free memory: never the queue that signals fences.
+	 */
+	struct workqueue_struct *free_wq;
+	/** @poll_work: Checks completions while jobs are out. */
+	struct delayed_work poll_work;
+
+	/** @reset_count: GPU resets since load. */
+	u32 reset_count;
+	/** @reset_logged: @reset_count at the last "started" message (resumes are quiet). */
+	u32 reset_logged;
+
+	/** @pb: The parameter buffer every render context shares (pvrsgx_render.c). */
+	struct pvrsgx_pb *pb;
+	/** @pb_lock: Serializes @pb's creation. */
+	struct mutex pb_lock;
+
+	/** @kobj_lock: Protects @krange, @graveyard and @kobj_unmapped. */
+	struct mutex kobj_lock;
+	/** @krange: Allocators of the device-wide kernel object ranges. */
+	struct drm_mm krange[PVRSGX_KRANGE_COUNT];
+	/** @graveyard: Released kernel objects the firmware may still read. */
+	struct list_head graveyard;
+	/** @kobj_unmapped: Unmapped kernel objects waiting for a BIF TLB invalidate. */
+	struct list_head kobj_unmapped;
+	/** @renders_submitted: Render jobs handed to the firmware, for reaping @graveyard. */
+	u64 renders_submitted;
+	/**
+	 * @done_seq: Jobs signalled since the microkernel started, against its
+	 * count (pvrsgx_fw_ta3d_ctl.done_seq; under @fence_lock).
+	 */
+	u32 done_seq;
+	/** @restarts: Microkernel restarts; graveyard entries from before one can go. */
+	u32 restarts;
+	/** @coredump_work: Hands a hang's dump to devcoredump, outside the reset. */
+	struct work_struct coredump_work;
+	/** @coredump: The dump waiting for @coredump_work, or NULL (@fence_lock). */
+	void *coredump;
+	/** @coredump_len: Its length. */
+	size_t coredump_len;
+	/** @renders_done: Render jobs the firmware has finished. */
+	u64 renders_done;
+	/** @reap_work: Frees @graveyard entries a later render made safe. */
+	struct work_struct reap_work;
+
+	/** @irq_count: SW_EVENT interrupts taken (debugfs). */
+	atomic_t irq_count;
+	/** @pm_pinned: Runtime PM is held active (single-context bring-up). */
+	bool pm_pinned;
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+	/** @test_wr: KUnit only: sees every register write before it lands. */
+	void (*test_wr)(struct pvrsgx_device *sdev, u32 reg, u32 val);
+#endif
+};
+
+#define to_pvrsgx_device(drm) container_of_const(drm, struct pvrsgx_device, base)
+
+/*
+ * A file's limits: every context and RT dataset takes kernel memory from
+ * device-wide ranges, so one client may not take them all.
+ */
+#define PVRSGX_MAX_FILE_CONTEXTS	64
+#define PVRSGX_MAX_FILE_RTDS		64
+
+/**
+ * struct pvrsgx_file - Per-open state
+ */
+struct pvrsgx_file {
+	/** @sdev: The device. */
+	struct pvrsgx_device *sdev;
+	/** @vms: VM contexts by handle. */
+	struct xarray vms;
+	/** @contexts: Contexts by handle. */
+	struct xarray contexts;
+	/** @rtds: RT datasets by handle. */
+	struct xarray rtds;
+	/** @usage: GPU time, for fdinfo. */
+	struct pvrsgx_usage *usage;
+};
+
+static inline struct pvrsgx_file *to_pvrsgx_file(struct drm_file *file)
+{
+	return file->driver_priv;
+}
+
+static inline u32 sgx_rd(struct pvrsgx_device *sdev, u32 reg)
+{
+	return ioread32(sdev->regs + reg);
+}
+
+static inline void sgx_wr(struct pvrsgx_device *sdev, u32 reg, u32 val)
+{
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+	if (sdev->test_wr)
+		sdev->test_wr(sdev, reg, val);
+#endif
+	iowrite32(val, sdev->regs + reg);
+}
+
+/*
+ * What list 0 holds while no job runs: an empty directory, or on a
+ * single-context core the firmware's, since the microkernel translates
+ * through list 0 too.
+ */
+static inline struct pvrsgx_mmu *pvrsgx_idle_mmu(const struct pvrsgx_device *sdev)
+{
+	return pvrsgx_core_single_context(sdev->family) ? sdev->kernel_mmu : sdev->idle_mmu;
+}
+
+/* The core's device-virtual layout. */
+static inline const struct pvrsgx_heap_layout *pvrsgx_heaps(const struct pvrsgx_device *sdev)
+{
+	return sdev->family->heaps;
+}
+
+/* A register from the core table: 0 is one the core lacks, never written. */
+static inline void sgx_wr_opt(struct pvrsgx_device *sdev, u32 reg, u32 val)
+{
+	if (reg)
+		sgx_wr(sdev, reg, val);
+}
+
+/*
+ * The microkernel translates through the last directory list; TA, 3D and
+ * host requests through list 0, which the microkernel points at the job's
+ * page directory.
+ */
+static inline u32 pvrsgx_dirlist_edm(const struct pvrsgx_device *sdev)
+{
+	return sdev->family->num_dir_lists - 1;
+}
+
+int pvrsgx_kmem_alloc(struct pvrsgx_device *sdev, struct pvrsgx_kmem *km,
+		      u32 dev_addr, u32 size, u32 pte_flags, gfp_t gfp);
+void pvrsgx_kmem_free(struct pvrsgx_device *sdev, struct pvrsgx_kmem *km);
+void pvrsgx_kmem_release(struct pvrsgx_kmem *km);
+void pvrsgx_share_fw_heaps(struct pvrsgx_device *sdev, struct pvrsgx_mmu *mmu);
+
+int pvrsgx_device_init(struct pvrsgx_device *sdev);
+void pvrsgx_device_unplug(struct pvrsgx_device *sdev);
+void pvrsgx_reset(struct pvrsgx_device *sdev, bool recovery);
+int pvrsgx_poll(struct pvrsgx_device *sdev, const u32 *addr, u32 mask, u32 val,
+		unsigned int timeout_us);
+
+#endif /* PVRSGX_DEVICE_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_hw.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_hw.h
new file mode 100644
index 000000000..acb0c09d8
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_hw.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_HW_H
+#define PVRSGX_HW_H
+
+#include <linux/bits.h>
+#include <linux/types.h>
+
+#include "pvrsgx_regs.h"
+
+/* Gating modes of the 2-bit CLKGATECTL/CLKGATECTL2 fields. */
+#define SGX_CLKG_ON				1
+#define SGX_CLKG_AUTO				2
+/* The same mode in every 2-bit field of mask. */
+#define SGX_CLKG_ALL(mask, mode)		((mask) / 3 * (mode))
+
+/*
+ * SGX545 registers not in the GPL header; the names are the driver's, from
+ * what the microkernel and the init script do with them. Code reaches them
+ * through the core table.
+ */
+#define SGX545_CR_EVENT_PDS_ENABLE		0x0a58	/* host events start the event program */
+#define   SGX545_CR_EVENT_PDS_ENABLE_ALL	0x2204b00f
+#define SGX545_CR_EVENT_PDS_BASE		0x0a68	/* the event PDS program, 16-byte aligned */
+#define SGX545_CR_EVENT_PDS_DATA0		0x0a6c
+#define SGX545_CR_EVENT_PDS_DATA1		0x0a70
+#define SGX545_CR_EDM_TASK_REG0			0x0b14	/* where the microkernel is */
+#define SGX545_CR_EDM_TASK_REG1			0x0b20
+
+/* Directory lists 1 and up are consecutive; 0 sits apart. */
+#define SGX_BIF_MAX_DIRLISTS			16
+
+static inline u32 sgx_cr_bif_dir_list_base(unsigned int n)
+{
+	return n ? SGX_CR_BIF_DIR_LIST_BASE1 + 4 * (n - 1) : SGX_CR_BIF_DIR_LIST_BASE0;
+}
+
+/* MMU: two levels, 4 KiB pages, 32-bit device addresses and entries. */
+#define SGX_MMU_PAGE_SHIFT			12
+#define SGX_MMU_PAGE_SIZE			BIT(SGX_MMU_PAGE_SHIFT)
+#define SGX_MMU_PT_SHIFT			10
+#define SGX_MMU_PD_ENTRIES			1024
+#define SGX_MMU_PT_ENTRIES			1024
+#define SGX_MMU_PDE_SPAN			BIT(SGX_MMU_PAGE_SHIFT + SGX_MMU_PT_SHIFT)
+#define SGX_MMU_ADDR_MASK			GENMASK(31, 12)
+#define SGX_MMU_PDE_VALID			BIT(0)
+#define SGX_MMU_PDE_PAGE_SIZE_4K		0
+#define SGX_MMU_PTE_VALID			BIT(0)
+#define SGX_MMU_PTE_WRITEONLY			BIT(1)
+#define SGX_MMU_PTE_READONLY			BIT(2)
+#define SGX_MMU_PTE_CACHECONSISTENT		BIT(3)
+#define SGX_MMU_PTE_EDMPROTECT			BIT(4)
+
+#endif /* PVRSGX_HW_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.c
new file mode 100644
index 000000000..7f204e7bb
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.c
@@ -0,0 +1,281 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * SGX MMU page tables. One page directory of 1024 entries, each naming a page
+ * table of 1024 four-KiB pages, so 4 GiB of device address space. Entries are
+ * the physical address with flags in the low 12 bits, so every table and every
+ * page the GPU sees must sit below 4 GiB.
+ *
+ * Tables are write-combined: whether the BIF snoops the CPU caches is not
+ * known. Callers order the writes before the GPU uses them by kicking the
+ * firmware, which issues a write barrier first.
+ */
+
+#include <linux/io.h>
+#include <linux/mm.h>
+#include <linux/mutex.h>
+#include <linux/scatterlist.h>
+#include <linux/slab.h>
+
+#include "pvrsgx_device.h"
+#include "pvrsgx_mmu.h"
+
+struct pvrsgx_mmu_pt {
+	struct page *page;
+	u32 *cpu;
+	unsigned int used;
+};
+
+struct pvrsgx_mmu {
+	struct pvrsgx_device *sdev;
+	/** @lock: Guards the directory, the page tables and their use counts. */
+	struct mutex lock;
+	struct page *pd_page;
+	u32 *pd;
+	struct pvrsgx_mmu_pt *pt[SGX_MMU_PD_ENTRIES];
+};
+
+static struct page *table_page_alloc(struct pvrsgx_device *sdev, u32 **cpu)
+{
+	struct page *p = alloc_page(GFP_KERNEL | __GFP_ZERO | __GFP_DMA32);
+
+	if (!p)
+		return NULL;
+	if (pvrsgx_pages_wc(sdev->base.dev, &p, 1))
+		goto err_free;
+	*cpu = pvrsgx_table_map(p);
+	if (!*cpu)
+		goto err_wb;
+	return p;
+
+err_wb:
+	pvrsgx_pages_wb(&p, 1);
+err_free:
+	__free_page(p);
+	return NULL;
+}
+
+static void table_page_free(struct page *p, u32 *cpu)
+{
+	pvrsgx_table_unmap(cpu);
+	pvrsgx_pages_wb(&p, 1);
+	__free_page(p);
+}
+
+struct pvrsgx_mmu *pvrsgx_mmu_create(struct pvrsgx_device *sdev)
+{
+	struct pvrsgx_mmu *mmu = kzalloc_obj(*mmu);
+
+	if (!mmu)
+		return ERR_PTR(-ENOMEM);
+
+	mmu->pd_page = table_page_alloc(sdev, &mmu->pd);
+	if (!mmu->pd_page) {
+		kfree(mmu);
+		return ERR_PTR(-ENOMEM);
+	}
+	mmu->sdev = sdev;
+	mutex_init(&mmu->lock);
+	return mmu;
+}
+
+void pvrsgx_mmu_destroy(struct pvrsgx_mmu *mmu)
+{
+	if (IS_ERR_OR_NULL(mmu))
+		return;
+
+	for (unsigned int i = 0; i < SGX_MMU_PD_ENTRIES; i++) {
+		if (!mmu->pt[i])
+			continue;
+		WARN_ON(mmu->pt[i]->used);
+		table_page_free(mmu->pt[i]->page, mmu->pt[i]->cpu);
+		kfree(mmu->pt[i]);
+	}
+	table_page_free(mmu->pd_page, mmu->pd);
+	mutex_destroy(&mmu->lock);
+	kfree(mmu);
+}
+
+u32 pvrsgx_mmu_pd_addr(struct pvrsgx_mmu *mmu)
+{
+	return lower_32_bits(page_to_phys(mmu->pd_page));
+}
+
+static u32 *pte_get(struct pvrsgx_mmu *mmu, u32 dev_addr, bool alloc)
+{
+	unsigned int pdi = dev_addr >> (SGX_MMU_PAGE_SHIFT + SGX_MMU_PT_SHIFT);
+	unsigned int pti = (dev_addr >> SGX_MMU_PAGE_SHIFT) & (SGX_MMU_PT_ENTRIES - 1);
+	struct pvrsgx_mmu_pt *pt = mmu->pt[pdi];
+
+	if (!pt) {
+		/* A directory entry with no table of ours is another directory's (shared). */
+		if (!alloc || WARN_ON(mmu->pd[pdi]))
+			return NULL;
+		pt = kzalloc_obj(*pt);
+		if (!pt)
+			return NULL;
+		pt->page = table_page_alloc(mmu->sdev, &pt->cpu);
+		if (!pt->page) {
+			kfree(pt);
+			return NULL;
+		}
+		mmu->pt[pdi] = pt;
+		mmu->pd[pdi] = (lower_32_bits(page_to_phys(pt->page)) & SGX_MMU_ADDR_MASK) |
+			       SGX_MMU_PDE_PAGE_SIZE_4K | SGX_MMU_PDE_VALID;
+	}
+	return &pt->cpu[pti];
+}
+
+static int pte_set(struct pvrsgx_mmu *mmu, u32 dev_addr, phys_addr_t pa, u32 flags)
+{
+	u32 *pte;
+
+	if (upper_32_bits(pa))
+		return -EFAULT;
+	pte = pte_get(mmu, dev_addr, true);
+	if (!pte)
+		return -ENOMEM;
+	if (WARN_ON(*pte & SGX_MMU_PTE_VALID))
+		return -EEXIST;
+	*pte = (lower_32_bits(pa) & SGX_MMU_ADDR_MASK) | flags | SGX_MMU_PTE_VALID;
+	mmu->pt[dev_addr >> (SGX_MMU_PAGE_SHIFT + SGX_MMU_PT_SHIFT)]->used++;
+	return 0;
+}
+
+static void unmap_locked(struct pvrsgx_mmu *mmu, u32 dev_addr, u64 size)
+{
+	for (u64 off = 0; off < size; off += SGX_MMU_PAGE_SIZE) {
+		u32 a = dev_addr + off;
+		u32 *pte = pte_get(mmu, a, false);
+
+		if (!pte || !(*pte & SGX_MMU_PTE_VALID))
+			continue;
+		*pte = 0;
+		mmu->pt[a >> (SGX_MMU_PAGE_SHIFT + SGX_MMU_PT_SHIFT)]->used--;
+	}
+}
+
+int pvrsgx_mmu_map_pages(struct pvrsgx_mmu *mmu, u32 dev_addr,
+			 struct page **pages, unsigned int npages, u32 flags)
+{
+	int err = 0;
+	unsigned int i;
+
+	mutex_lock(&mmu->lock);
+	for (i = 0; i < npages; i++) {
+		err = pte_set(mmu, dev_addr + i * SGX_MMU_PAGE_SIZE,
+			      page_to_phys(pages[i]), flags);
+		if (err)
+			break;
+	}
+	if (err)
+		unmap_locked(mmu, dev_addr, (u64)i * SGX_MMU_PAGE_SIZE);
+	mutex_unlock(&mmu->lock);
+	return err;
+}
+
+int pvrsgx_mmu_map_sgt(struct pvrsgx_mmu *mmu, u32 dev_addr,
+		       struct sg_table *sgt, u64 offset, u64 size, u32 flags)
+{
+	struct sg_dma_page_iter it;
+	u64 skip = offset >> PAGE_SHIFT, done = 0;
+	int err = 0;
+
+	mutex_lock(&mmu->lock);
+	for_each_sgtable_dma_page(sgt, &it, skip) {
+		if (done >= size)
+			break;
+		err = pte_set(mmu, dev_addr + done, sg_page_iter_dma_address(&it), flags);
+		if (err)
+			break;
+		done += SGX_MMU_PAGE_SIZE;
+	}
+	if (!err && done < size)
+		err = -EINVAL;
+	if (err)
+		unmap_locked(mmu, dev_addr, done);
+	mutex_unlock(&mmu->lock);
+	return err;
+}
+
+/*
+ * Allocate the page tables of [dev_addr, dev_addr + size), so that a later map
+ * of the range can't fail for memory. Tables stay until the directory goes.
+ */
+int pvrsgx_mmu_reserve(struct pvrsgx_mmu *mmu, u32 dev_addr, u64 size)
+{
+	u64 span = SGX_MMU_PDE_SPAN, end = (u64)dev_addr + size;
+	int err = 0;
+
+	mutex_lock(&mmu->lock);
+	for (u64 a = round_down(dev_addr, span); a < end && !err; a += span)
+		if (!pte_get(mmu, a, true))
+			err = -ENOMEM;
+	mutex_unlock(&mmu->lock);
+	return err;
+}
+
+void pvrsgx_mmu_unmap(struct pvrsgx_mmu *mmu, u32 dev_addr, u64 size)
+{
+	mutex_lock(&mmu->lock);
+	unmap_locked(mmu, dev_addr, size);
+	mutex_unlock(&mmu->lock);
+}
+
+bool pvrsgx_mmu_range_empty(struct pvrsgx_mmu *mmu, u32 dev_addr, u64 size)
+{
+	bool empty = true;
+
+	mutex_lock(&mmu->lock);
+	for (u64 off = 0; off < size && empty; off += SGX_MMU_PAGE_SIZE) {
+		u32 *pte = pte_get(mmu, dev_addr + off, false);
+
+		empty = !pte || !(*pte & SGX_MMU_PTE_VALID);
+	}
+	mutex_unlock(&mmu->lock);
+	return empty;
+}
+
+/*
+ * For a hang dump: the directory and table entries for dev_addr, or false if
+ * the directory is busy. Never allocates.
+ */
+bool pvrsgx_mmu_lookup(struct pvrsgx_mmu *mmu, u32 dev_addr, u32 *pde, u32 *pte)
+{
+	u32 *p;
+
+	if (!mutex_trylock(&mmu->lock))
+		return false;
+	*pde = mmu->pd[dev_addr >> (SGX_MMU_PAGE_SHIFT + SGX_MMU_PT_SHIFT)];
+	p = pte_get(mmu, dev_addr, false);
+	*pte = p ? *p : 0;
+	mutex_unlock(&mmu->lock);
+	return true;
+}
+
+/* Give [start, end) page tables of their own now, so their PDEs never change. */
+int pvrsgx_mmu_prealloc(struct pvrsgx_mmu *mmu, u32 start, u64 end)
+{
+	int err = 0;
+
+	mutex_lock(&mmu->lock);
+	for (u64 a = start; a < end && !err; a += SGX_MMU_PDE_SPAN)
+		err = pte_get(mmu, a, true) ? 0 : -ENOMEM;
+	mutex_unlock(&mmu->lock);
+	return err;
+}
+
+/*
+ * Point dst's directory entries for [start, end) at src's page tables: src's
+ * mappings there show in dst too, and dst never maps anything there itself.
+ * Both ends 4 MiB aligned, src preallocated.
+ */
+void pvrsgx_mmu_share(struct pvrsgx_mmu *dst, struct pvrsgx_mmu *src, u32 start, u64 end)
+{
+	for (u64 a = start; a < end; a += SGX_MMU_PDE_SPAN) {
+		unsigned int i = a >> (SGX_MMU_PAGE_SHIFT + SGX_MMU_PT_SHIFT);
+
+		dst->pd[i] = src->pd[i];
+	}
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.h
new file mode 100644
index 000000000..7ad63a4db
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_mmu.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_MMU_H
+#define PVRSGX_MMU_H
+
+#include <linux/types.h>
+
+struct page;
+struct pvrsgx_device;
+struct sg_table;
+
+struct pvrsgx_mmu *pvrsgx_mmu_create(struct pvrsgx_device *sdev);
+void pvrsgx_mmu_destroy(struct pvrsgx_mmu *mmu);
+u32 pvrsgx_mmu_pd_addr(struct pvrsgx_mmu *mmu);
+
+int pvrsgx_mmu_map_pages(struct pvrsgx_mmu *mmu, u32 dev_addr,
+			 struct page **pages, unsigned int npages, u32 flags);
+int pvrsgx_mmu_map_sgt(struct pvrsgx_mmu *mmu, u32 dev_addr,
+		       struct sg_table *sgt, u64 offset, u64 size, u32 flags);
+int pvrsgx_mmu_reserve(struct pvrsgx_mmu *mmu, u32 dev_addr, u64 size);
+void pvrsgx_mmu_unmap(struct pvrsgx_mmu *mmu, u32 dev_addr, u64 size);
+bool pvrsgx_mmu_range_empty(struct pvrsgx_mmu *mmu, u32 dev_addr, u64 size);
+int pvrsgx_mmu_prealloc(struct pvrsgx_mmu *mmu, u32 start, u64 end);
+void pvrsgx_mmu_share(struct pvrsgx_mmu *dst, struct pvrsgx_mmu *src, u32 start, u64 end);
+bool pvrsgx_mmu_lookup(struct pvrsgx_mmu *mmu, u32 dev_addr, u32 *pde, u32 *pte);
+
+#endif /* PVRSGX_MMU_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_regs.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_regs.h
new file mode 100644
index 000000000..21e5e2098
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_regs.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) Imagination Technologies Ltd. All rights reserved.
+ * Copyright © 2026 Joseph Telaak
+ *
+ * The registers and fields the driver uses, generated by gen_regs.py from
+ * Imagination's sgx545defs.h, which is GPL-2.0; both live in the driver's
+ * source repository, https://github.com/The1TrueJoe/powervr_sgx.
+ */
+
+#ifndef PVRSGX_REGS_H
+#define PVRSGX_REGS_H
+
+#include <linux/bits.h>
+
+/*
+ * SGX5xx core registers, from BAR offset 0. On a core nobody has initialized
+ * only CLKGATECTL, CLKGATESTATUS, SOFT_RESET, EVENT_STATUS, EVENT_HOST_ENABLE,
+ * BIF_CTRL, CORE_ID and CORE_REVISION are safe to read; some others hang the
+ * bus until the reset sequence has run.
+ */
+#define SGX_CR_CLKGATECTL			0x0000
+#define   SGX_CR_CLKGATECTL_DPM_CLKG		GENMASK(11, 10)
+#define   SGX_CR_CLKGATECTL_UNITS		GENMASK(15, 0)
+#define SGX_CR_CLKGATECTL2			0x0004
+#define   SGX_CR_CLKGATECTL2_UNITS		GENMASK(23, 0)
+#define SGX_CR_CLKGATESTATUS			0x0008
+#define   SGX_CR_CLKGATESTATUS_DPM_CLKS		BIT(5)
+#define SGX_CR_CORE_ID				0x001c
+#define SGX_CR_CORE_REVISION			0x0020
+#define SGX_CR_SOFT_RESET			0x0080
+#define   SGX_CR_SOFT_RESET_BIF_RESET		BIT(0)
+#define   SGX_CR_SOFT_RESET_DPM_RESET		BIT(1)
+#define   SGX_CR_SOFT_RESET_TA_RESET		BIT(2)
+#define   SGX_CR_SOFT_RESET_USE_RESET		BIT(3)
+#define   SGX_CR_SOFT_RESET_ISP_RESET		BIT(4)
+#define   SGX_CR_SOFT_RESET_TSP_RESET		BIT(5)
+#define SGX_CR_EVENT_HOST_ENABLE2		0x0110
+#define SGX_CR_EVENT_HOST_CLEAR2		0x0114
+#define SGX_CR_EVENT_STATUS2			0x0118
+#define SGX_CR_EVENT_STATUS			0x012c
+#define   SGX_CR_EVENT_STATUS_MASTER_INTERRUPT	BIT(31)
+#define   SGX_CR_EVENT_STATUS_TIMER		BIT(29)
+#define   SGX_CR_EVENT_STATUS_SW_EVENT		BIT(14)
+#define SGX_CR_EVENT_HOST_ENABLE		0x0130
+#define SGX_CR_EVENT_HOST_CLEAR			0x0134
+#define   SGX_CR_EVENT_HOST_CLEAR_MASTER_INTERRUPT	BIT(31)
+#define   SGX_CR_EVENT_HOST_CLEAR_SW_EVENT	BIT(14)
+#define SGX_CR_TIMER				0x0144
+#define SGX_CR_USE_CODE_BASE_0			0x0a0c
+#define SGX_CR_PDS_EXEC_BASE			0x0ab8
+#define SGX_CR_EVENT_KICKER			0x0ac4
+#define SGX_CR_EVENT_KICK			0x0ac8
+#define   SGX_CR_EVENT_KICK_NOW			BIT(0)
+#define SGX_CR_EVENT_TIMER			0x0acc
+#define SGX_CR_EVENT_KICK2			0x0ae8
+#define   SGX_CR_EVENT_KICK2_NOW		BIT(0)
+#define SGX_CR_BIF_CTRL				0x0c00
+#define   SGX_CR_BIF_CTRL_PAUSE			BIT(1)
+#define   SGX_CR_BIF_CTRL_INVALDC		BIT(3)
+#define   SGX_CR_BIF_CTRL_CLEAR_FAULT		BIT(4)
+#define SGX_CR_BIF_INT_STAT			0x0c04
+#define   SGX_CR_BIF_INT_STAT_FAULT_REQ		GENMASK(15, 0)
+#define   SGX_CR_BIF_INT_STAT_FAULT_TYPE	GENMASK(18, 16)
+#define   SGX_CR_BIF_INT_STAT_FLUSH_COMPLETE	BIT(19)
+#define SGX_CR_BIF_FAULT			0x0c08
+#define SGX_CR_BIF_DIR_LIST_BASE0		0x0c84
+#define SGX_CR_BIF_DIR_LIST_BASE1		0x0c38
+#define SGX_CR_BIF_BANK_SET			0x0c74
+#define SGX_CR_BIF_BANK0			0x0c78
+#define   SGX_CR_BIF_BANK0_INDEX_EDM		GENMASK(3, 0)
+#define   SGX_CR_BIF_BANK0_INDEX_TA		GENMASK(7, 4)
+#define   SGX_CR_BIF_BANK0_INDEX_HOST		GENMASK(11, 8)
+#define   SGX_CR_BIF_BANK0_INDEX_3D		GENMASK(15, 12)
+#define   SGX_CR_BIF_BANK0_INDEX_2D		GENMASK(19, 16)
+#define SGX_CR_BIF_MEM_REQ_STAT			0x0ca8
+#define   SGX_CR_BIF_MEM_REQ_STAT_READS		GENMASK(10, 0)
+
+#endif /* PVRSGX_REGS_H */

-- 
2.50.1

[PATCH RFC 06/13] drm/powervr_sgx: Load and start the microkernel

[Joseph Telaak]

The GPU runs a microkernel on its shader unit. Load it with
request_firmware(), check its header (magic, interface version,
features, image CRC, entry points and patch sites), and refuse an image
that doesn't declare secure mode, in which no client page directory maps
the firmware's memory.

Build the microkernel's control blocks and entry PDS programs in kernel
memory, run the core's init script, kick the first event and wait for
the microkernel to report it is running. Stopping is the microkernel's
power-off, then the cores held in reset.

Commands reach the microkernel through a kernel-owned ring (the kernel
CCB); a command is done when the microkernel's read offset passes it.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c  | 204 +++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.h  |  52 +++
 drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c   | 731 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_fw.h   |  77 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_fwif.h | 519 +++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_pds.c  |  97 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_pds.h  |  37 ++
 7 files changed, 1717 insertions(+)

diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c
new file mode 100644
index 000000000..351a7f930
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The kernel CCB: a ring of 256 commands the host writes and the microkernel
+ * reads. The host advances the write offset and rings
+ * EUR_CR_EVENT_KICK; the microkernel advances the read offset as it takes each
+ * command. Neither raises an interrupt by itself, so a command is complete, as
+ * far as the ring is concerned, once the read offset has passed it.
+ */
+
+#include <drm/drm_print.h>
+
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/jiffies.h>
+#include <linux/string.h>
+
+#include "pvrsgx_ccb.h"
+#include "pvrsgx_device.h"
+
+#define CCB_WO 0
+#define CCB_RO 1
+
+static u32 *ccbctl(struct pvrsgx_device *sdev)
+{
+	return sdev->fw.cpu[PVRSGX_FW_CCBCTL];
+}
+
+void pvrsgx_ccb_reset(struct pvrsgx_device *sdev)
+{
+	memset(sdev->fw.cpu[PVRSGX_FW_CCB], 0, PVRSGX_CCB_SLOTS * PVRSGX_CCB_CMD_WORDS * 4);
+	ccbctl(sdev)[CCB_WO] = 0;
+	ccbctl(sdev)[CCB_RO] = 0;
+	*(u32 *)sdev->fw.cpu[PVRSGX_FW_KICKER] = 0;
+	/*
+	 * @ccb_issued carries on: a job sent after a restart has a higher sequence
+	 * than any count read before it, so a stale count can't complete it.
+	 */
+}
+
+/* Commands the microkernel has taken off the ring since the driver loaded. */
+u64 pvrsgx_ccb_consumed(struct pvrsgx_device *sdev)
+{
+	u32 ro = READ_ONCE(ccbctl(sdev)[CCB_RO]);
+	unsigned long flags;
+	u64 consumed;
+
+	/* The read offset is GPU-written: trust it only as an index. */
+	spin_lock_irqsave(&sdev->ccb_spin, flags);
+	consumed = sdev->ccb_issued - ((ccbctl(sdev)[CCB_WO] - ro) & (PVRSGX_CCB_SLOTS - 1));
+	spin_unlock_irqrestore(&sdev->ccb_spin, flags);
+	return consumed;
+}
+
+/* Write one command and ring the doorbell, with ccb_lock held. */
+static int ccb_write(struct pvrsgx_device *sdev, enum pvrsgx_kcmd type, u32 cache_control,
+		     const u32 data[6], u64 *seq)
+{
+	u32 *ctl = ccbctl(sdev), *cmd, wo;
+	unsigned long timeout = jiffies + msecs_to_jiffies(500);
+
+	wo = ctl[CCB_WO];
+	while (((wo + 1) & (PVRSGX_CCB_SLOTS - 1)) ==
+	       (READ_ONCE(ctl[CCB_RO]) & (PVRSGX_CCB_SLOTS - 1))) {
+		if (time_after(jiffies, timeout)) {
+			drm_err(&sdev->base, "kernel CCB full (wo %u ro %u)\n", wo, ctl[CCB_RO]);
+			return -EBUSY;
+		}
+		usleep_range(500, 1000);
+	}
+
+	cmd = (u32 *)sdev->fw.cpu[PVRSGX_FW_CCB] + wo * PVRSGX_CCB_CMD_WORDS;
+	cmd[0] = type;
+	cmd[1] = cache_control;
+	memcpy(&cmd[2], data, 6 * sizeof(u32));
+
+	/* Command, then offset, then doorbell. */
+	wmb();
+	spin_lock_irq(&sdev->ccb_spin);
+	ctl[CCB_WO] = (wo + 1) & (PVRSGX_CCB_SLOTS - 1);
+	sdev->ccb_issued++;
+	if (seq)
+		*seq = sdev->ccb_issued;
+	spin_unlock_irq(&sdev->ccb_spin);
+	*(u32 *)sdev->fw.cpu[PVRSGX_FW_KICKER] =
+		(*(u32 *)sdev->fw.cpu[PVRSGX_FW_KICKER] + 1) & 0xff;
+	/* The command and the kicker count reach memory before the kick. */
+	wmb();
+	sgx_wr(sdev, SGX_CR_EVENT_KICK, SGX_CR_EVENT_KICK_NOW);
+	return 0;
+}
+
+/*
+ * BRN_28889: a BIF directory or table invalidate and a data invalidate never
+ * go in one command. The BIF part goes first, alone, and the microkernel must
+ * have taken it before the data part is sent. An invalidate waiter
+ * (pvrsgx_ccb_after_invalidate()) waits for the first, which carries the BIF
+ * part.
+ */
+static int ccb_split_inval(struct pvrsgx_device *sdev, u32 *cache_control)
+{
+	static const u32 none[6];
+	unsigned long timeout = jiffies + msecs_to_jiffies(500);
+	u64 seq;
+	int err;
+
+	if (!(sdev->family->quirks & PVRSGX_QUIRK_BRN_28889) ||
+	    !(*cache_control & PVRSGX_CC_INVAL_DATA) ||
+	    !(*cache_control & ~PVRSGX_CC_INVAL_DATA))
+		return 0;
+	err = ccb_write(sdev, PVRSGX_KCMD_PROCESS_QUEUES, *cache_control & ~PVRSGX_CC_INVAL_DATA,
+			none, &seq);
+	if (err)
+		return err;
+	while (pvrsgx_ccb_consumed(sdev) < seq) {
+		if (time_after(jiffies, timeout)) {
+			drm_err(&sdev->base, "BIF invalidate not taken (BRN_28889)\n");
+			return -ETIMEDOUT;
+		}
+		usleep_range(100, 200);
+	}
+	*cache_control = PVRSGX_CC_INVAL_DATA;
+	return 0;
+}
+
+int pvrsgx_ccb_send(struct pvrsgx_device *sdev, enum pvrsgx_kcmd type,
+		    const u32 data[6], u64 *seq)
+{
+	u32 cache_control;
+	int err;
+
+	/* Types past the microkernel's command table are refused. */
+	if (type >= PVRSGX_KCMD_MAX)
+		return -EINVAL;
+
+	mutex_lock(&sdev->ccb_lock);
+	cache_control = sdev->fw.cache_control;
+	err = ccb_split_inval(sdev, &cache_control);
+	if (!err)
+		err = ccb_write(sdev, type, cache_control, data, seq);
+	if (!err)
+		sdev->fw.cache_control = 0;
+	mutex_unlock(&sdev->ccb_lock);
+	return err;
+}
+
+void pvrsgx_ccb_after_invalidate(struct pvrsgx_device *sdev, struct pvrsgx_inval_wait *w,
+				 u32 cache_control)
+{
+	unsigned long flags;
+
+	if (READ_ONCE(sdev->lost)) {
+		w->done(w);
+		return;
+	}
+	mutex_lock(&sdev->ccb_lock);
+	sdev->fw.cache_control |= cache_control;
+	w->seq = sdev->ccb_issued + 1;
+	mutex_unlock(&sdev->ccb_lock);
+	spin_lock_irqsave(&sdev->inval_lock, flags);
+	list_add_tail(&w->node, &sdev->inval_waits);
+	spin_unlock_irqrestore(&sdev->inval_lock, flags);
+}
+
+/* For status dumps: how much work still waits for an invalidate. */
+u32 pvrsgx_ccb_inval_pending(struct pvrsgx_device *sdev)
+{
+	struct pvrsgx_inval_wait *w;
+	unsigned long flags;
+	u32 n = 0;
+
+	spin_lock_irqsave(&sdev->inval_lock, flags);
+	list_for_each_entry(w, &sdev->inval_waits, node)
+		n++;
+	spin_unlock_irqrestore(&sdev->inval_lock, flags);
+	return n;
+}
+
+/*
+ * Run what waited for an invalidate the microkernel has now taken; all of it
+ * once the GPU is stopped. A restart empties the ring, so everything sent
+ * before it counts as taken: the GPU was reset, and the first command after
+ * the start carries an invalidate of its own.
+ */
+void pvrsgx_ccb_inval_reap(struct pvrsgx_device *sdev, bool all)
+{
+	struct pvrsgx_inval_wait *w, *tmp;
+	unsigned long flags;
+	LIST_HEAD(done);
+	u64 consumed;
+
+	spin_lock_irqsave(&sdev->inval_lock, flags);
+	consumed = all || READ_ONCE(sdev->lost) ? U64_MAX : pvrsgx_ccb_consumed(sdev);
+	list_for_each_entry_safe(w, tmp, &sdev->inval_waits, node) {
+		if (w->seq <= consumed)
+			list_move_tail(&w->node, &done);
+	}
+	spin_unlock_irqrestore(&sdev->inval_lock, flags);
+	list_for_each_entry_safe(w, tmp, &done, node) {
+		list_del(&w->node);
+		w->done(w);
+	}
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.h
new file mode 100644
index 000000000..2968cb41b
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_CCB_H
+#define PVRSGX_CCB_H
+
+#include <linux/list.h>
+#include <linux/types.h>
+
+struct pvrsgx_device;
+
+/**
+ * struct pvrsgx_inval_wait - Work that waits for a BIF invalidate to run
+ * @node: In pvrsgx_device.inval_waits.
+ * @seq: The kernel CCB command that carries the invalidate.
+ * @done: Called, in process context, once the microkernel has taken it.
+ */
+struct pvrsgx_inval_wait {
+	struct list_head node;
+	u64 seq;
+	void (*done)(struct pvrsgx_inval_wait *w);
+};
+
+/* Kernel CCB command types. */
+enum pvrsgx_kcmd {
+	PVRSGX_KCMD_TA = 0,
+	PVRSGX_KCMD_TRANSFER = 1,
+	PVRSGX_KCMD_POWER = 3,
+	PVRSGX_KCMD_PROCESS_QUEUES = 7,
+	PVRSGX_KCMD_MAX = 11
+};
+
+/* Cache control: invalidates the microkernel runs before the command. */
+#define PVRSGX_CC_INVAL_BIF_PT	0x1
+#define PVRSGX_CC_INVAL_BIF_PD	0x2
+#define PVRSGX_CC_INVAL_BIF_SL	0x4
+#define PVRSGX_CC_INVAL_DATA	0x8
+
+/* A kernel CCB command: entry, cache control, six data words. */
+#define PVRSGX_CCB_SLOTS	256
+#define PVRSGX_CCB_CMD_WORDS	8
+
+int pvrsgx_ccb_send(struct pvrsgx_device *sdev, enum pvrsgx_kcmd type,
+		    const u32 data[6], u64 *seq);
+u64 pvrsgx_ccb_consumed(struct pvrsgx_device *sdev);
+void pvrsgx_ccb_reset(struct pvrsgx_device *sdev);
+void pvrsgx_ccb_after_invalidate(struct pvrsgx_device *sdev, struct pvrsgx_inval_wait *w,
+				 u32 cache_control);
+void pvrsgx_ccb_inval_reap(struct pvrsgx_device *sdev, bool all);
+u32 pvrsgx_ccb_inval_pending(struct pvrsgx_device *sdev);
+
+#endif /* PVRSGX_CCB_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c
new file mode 100644
index 000000000..b3c0d1891
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c
@@ -0,0 +1,731 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Loading and starting the microkernel: the buffers, constants, PDS programs
+ * and init scripts it needs.
+ *
+ * The host interface is version 1 of the microkernel's own
+ * (DRM_POWERVR_SGX_FW_IF_V1). The microkernel file's header lists its entry
+ * points and patch sites.
+ */
+
+#include <drm/drm_print.h>
+
+#include <linux/bitfield.h>
+#include <linux/crc32.h>
+#include <linux/delay.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/seq_buf.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_ccb.h"
+#include "pvrsgx_core.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_fwif.h"
+#include "pvrsgx_kobj.h"
+#include "pvrsgx_mmu.h"
+#include "pvrsgx_pds.h"
+
+/* The microkernel file's header. */
+#define UKFW_MAGIC0		0x35584753	/* "SGX5" */
+#define UKFW_MAGIC1		0x4e524b55	/* "UKRN" */
+#define UKFW_HEADER_MIN		0x60
+#define UKFW_IF_MAJOR		1
+#define UKFW_FEATURES		(PVRSGX_UKFW_CLEANUP_V2 | PVRSGX_UKFW_SECURE | \
+				 PVRSGX_UKFW_DONE_SEQ | PVRSGX_UKFW_SINGLE_CONTEXT)
+/* Bits 16-31: the microkernel's own build variants, nothing the host must know. */
+#define UKFW_FEATURES_PRIVATE	GENMASK(31, 16)
+#define UKFW_MAX_IMAGE		(CODE_ARENA_SIZE - PDS_BYTES - 0x10)
+
+struct ukfw_header {
+	__le32 magic[2];
+	__le32 header_size;
+	__le32 interface;
+	__le32 features;
+	__le32 image_size;
+	__le32 image_crc32;
+	__le32 n_entries;
+	__le32 entries_off;
+	__le32 n_patches;
+	__le32 patches_off;
+	__le32 core_rev;
+	__le32 reserved[4];
+	char build_id[32];
+};
+
+struct ukfw_pair {
+	__le32 a, b;
+};
+
+enum ukfw_patch_value {
+	UKFW_TA3DCTL = 1,
+	UKFW_HOSTCTL,
+	UKFW_CCBCTL,
+	UKFW_CCB,
+	UKFW_MISC,
+	UKFW_PDTABLE,	/* single-context cores only */
+};
+
+/* The POWER command: data word 1. */
+#define POWER_OFF			1
+
+/* The microkernel's timer runs at 1 kHz. */
+#define UKERNEL_TIMER_HZ	1000
+
+/*
+ * Where each buffer lives: its offset in the kernel data or code arena, and
+ * its size.
+ */
+static const struct {
+	bool code;
+	u32 offset, size;
+} layout[PVRSGX_FW_NBUF] = {
+	[PVRSGX_FW_CCB] = { false, 0x0000, 0x2000 },
+	[PVRSGX_FW_CCBCTL] = { false, 0x2000, 0x8 },
+	[PVRSGX_FW_KICKER] = { false, 0x2010, 0x4 },
+	[PVRSGX_FW_TA3DCTL] = { false, 0x3000, sizeof(struct pvrsgx_fw_ta3d_ctl) },
+	/* Right after the control block, which fills 0x3000-0x30ff. */
+	[PVRSGX_FW_HOSTCTL] = { false, 0x3100, 0x44 },
+	[PVRSGX_FW_MISC] = { false, 0x3160, 0x54 },
+	/* The image at 0, then the PDS programs from its end. */
+	[PVRSGX_FW_UKERNEL] = { true, 0x0000, 0 },
+	[PVRSGX_FW_PDS0] = { true, 0x000, 0x18 },
+	[PVRSGX_FW_PDS1] = { true, 0x020, 0x38 },
+	[PVRSGX_FW_PDS2] = { true, 0x060, 0x18 },
+	[PVRSGX_FW_PDS3] = { true, 0x090, 0x100 },
+	[PVRSGX_FW_PDS4] = { true, 0x190, 0x68 },
+};
+
+static_assert(0x3000 + sizeof(struct pvrsgx_fw_ta3d_ctl) <= 0x3100);
+
+#define PDS_BYTES	0x200
+
+#define DATA_ARENA_SIZE	0xe000
+#define CODE_ARENA_SIZE	0xe000
+
+static u32 *fw_words(struct pvrsgx_device *sdev, enum pvrsgx_fw_buf b)
+{
+	return sdev->fw.cpu[b];
+}
+
+/* The TA3D control block as the microkernel left it (in-flight slot, PB state). */
+void pvrsgx_fw_dump(struct seq_buf *sb, struct pvrsgx_device *sdev, unsigned int b,
+		    const char *name)
+{
+	const u32 *w = fw_words(sdev, b);
+
+	seq_buf_printf(sb, "%s @ 0x%08x:", name, sdev->fw.addr[b]);
+	for (u32 i = 0; i < layout[b].size / 4; i++)
+		seq_buf_printf(sb, "%s%08x", i % 8 ? " " : "\n  ", READ_ONCE(w[i]));
+	seq_buf_puts(sb, "\n");
+}
+
+u32 pvrsgx_fw_hostctl(struct pvrsgx_device *sdev, unsigned int word)
+{
+	return READ_ONCE(fw_words(sdev, PVRSGX_FW_HOSTCTL)[word]);
+}
+
+void pvrsgx_fw_hostctl_set(struct pvrsgx_device *sdev, unsigned int word, u32 v)
+{
+	WRITE_ONCE(fw_words(sdev, PVRSGX_FW_HOSTCTL)[word], v);
+}
+
+/*
+ * Fill one of the microkernel's limm placeholders (word 0 0x?dbeef, the value
+ * bits of word 1 clear): bits 0-20 to word 0, 21-25 to word 1 bits 4-8, 26-31
+ * to word 1 bits 12-17.
+ */
+static int fw_limm(struct pvrsgx_device *sdev, u32 off, u32 v)
+{
+	u32 *i = (u32 *)((u8 *)sdev->fw.cpu[PVRSGX_FW_UKERNEL] + off);
+	bool ok = (i[0] & 0x1fffff) == 0xdbeef && !(i[1] & 0x0003f1f0);
+
+	if (off > sdev->fw.image_size - 8 || !ok) {
+		drm_err(&sdev->base, "%s: no patch placeholder at +0x%x\n", sdev->fw.name, off);
+		return -EINVAL;
+	}
+	i[0] = (i[0] & 0xffe00000) | (v & 0x1fffff);
+	i[1] = (i[1] & 0xfffc0e0f) | ((v >> 17) & 0x1f0) | ((v >> 26) << 12);
+	return 0;
+}
+
+/*
+ * The microkernel's control block, rebuilt at every start: after a reset it
+ * must not find the previous run's render in flight.
+ */
+static void ta3dctl_fill(struct pvrsgx_device *sdev)
+{
+	struct pvrsgx_fw_ta3d_ctl *t = sdev->fw.cpu[PVRSGX_FW_TA3DCTL];
+	struct pvrsgx_fw *f = &sdev->fw;
+
+	memset(t, 0, sizeof(*t));
+	t->self = f->addr[PVRSGX_FW_TA3DCTL];
+	t->host_ctl = f->addr[PVRSGX_FW_HOSTCTL];
+	t->ccb_ctl = f->addr[PVRSGX_FW_CCBCTL];
+	t->ccb = f->addr[PVRSGX_FW_CCB];
+	t->kernel_pd = pvrsgx_mmu_pd_addr(sdev->kernel_mmu);
+	/* A fresh microkernel counts finished jobs from 0. */
+	WRITE_ONCE(sdev->done_seq, 0);
+}
+
+/*
+ * The open microkernel's entry programs and patches, from its file header.
+ * A PDS program it doesn't list enters where program 0 does: its event entry
+ * dispatches every event itself.
+ */
+static int fw_build_open(struct pvrsgx_device *sdev)
+{
+	static const enum pvrsgx_fw_buf value_buf[] = {
+		[UKFW_TA3DCTL] = PVRSGX_FW_TA3DCTL,
+		[UKFW_HOSTCTL] = PVRSGX_FW_HOSTCTL,
+		[UKFW_CCBCTL] = PVRSGX_FW_CCBCTL,
+		[UKFW_CCB] = PVRSGX_FW_CCB,
+		[UKFW_MISC] = PVRSGX_FW_MISC,
+	};
+	struct pvrsgx_fw *f = &sdev->fw;
+	u32 entry[5];
+	int err = 0;
+
+	/* The header check guarantees program 0 is listed. */
+	for (u32 i = 0; i < f->n_entries; i++)
+		if (!f->entries[i][0])
+			memset32(entry, f->entries[i][1], ARRAY_SIZE(entry));
+	for (u32 i = 0; i < f->n_entries; i++)
+		entry[f->entries[i][0]] = f->entries[i][1];
+	for (u32 p = 0; p < ARRAY_SIZE(entry); p++)
+		pvrsgx_pds_entry(sdev, PVRSGX_FW_PDS0 + p, entry[p]);
+
+	for (u32 i = 0; i < f->n_patches && !err; i++) {
+		u32 v = f->patches[i][1];
+
+		if (v == UKFW_PDTABLE && !f->pdtable.pages) {
+			drm_err(&sdev->base, "%s: a PD table patch on a multi-context core\n",
+				f->name);
+			return -EINVAL;
+		}
+		err = fw_limm(sdev, f->patches[i][0],
+			      v == UKFW_PDTABLE ? f->pdtable.dev_addr : f->addr[value_buf[v]]);
+	}
+	if (err)
+		return err;
+	return 0;
+}
+
+/* Upload and patch the microkernel and assemble its entry programs. */
+static int fw_build(struct pvrsgx_device *sdev, const u8 *image)
+{
+	struct pvrsgx_fw *f = &sdev->fw;
+
+	memcpy(f->cpu[PVRSGX_FW_UKERNEL], image, f->image_size);
+
+	return fw_build_open(sdev);
+}
+
+/* Copy n pairs at off of the file, or NULL if they run past its end. */
+static u32 (*ukfw_table(const u8 *data, size_t size, u32 off, u32 n))[2]
+{
+	const struct ukfw_pair *src = (const void *)(data + off);
+	u32 (*t)[2];
+
+	if (!n || n > 64 || off > size || n * sizeof(*src) > size - off || !IS_ALIGNED(off, 4))
+		return NULL;
+	t = kmalloc_array(n, sizeof(*t), GFP_KERNEL);
+	if (!t)
+		return NULL;
+	for (u32 i = 0; i < n; i++) {
+		t[i][0] = le32_to_cpu(src[i].a);
+		t[i][1] = le32_to_cpu(src[i].b);
+	}
+	return t;
+}
+
+/*
+ * The microkernel file: check the header and take
+ * its image and tables into f. -ENOENT if data is not one; on -EINVAL, *why
+ * says what is wrong. f's tables and image are the caller's to free either way.
+ */
+static int ukfw_parse(const u8 *data, size_t size, struct pvrsgx_fw *f, const char **why)
+{
+	const struct ukfw_header *h = (const void *)data;
+	u32 hsize, isize, iface;
+	bool entry0 = false;
+
+	if (size < UKFW_HEADER_MIN || le32_to_cpu(h->magic[0]) != UKFW_MAGIC0 ||
+	    le32_to_cpu(h->magic[1]) != UKFW_MAGIC1)
+		return -ENOENT;
+	hsize = le32_to_cpu(h->header_size);
+	isize = le32_to_cpu(h->image_size);
+	iface = le32_to_cpu(h->interface);
+	f->features = le32_to_cpu(h->features);
+	*why = "bad sizes";
+	if (hsize < UKFW_HEADER_MIN || !IS_ALIGNED(hsize, 8) || hsize > size ||
+	    !isize || !IS_ALIGNED(isize, 8) || isize > size - hsize || isize > UKFW_MAX_IMAGE)
+		return -EINVAL;
+	*why = "interface or features not supported";
+	if (iface >> 16 != UKFW_IF_MAJOR ||
+	    (f->features & ~(UKFW_FEATURES | UKFW_FEATURES_PRIVATE)))
+		return -EINVAL;
+	*why = "image CRC mismatch";
+	if (crc32_le(~0, data + hsize, isize) != ~le32_to_cpu(h->image_crc32))
+		return -EINVAL;
+
+	*why = "bad entry or patch table";
+	f->n_entries = le32_to_cpu(h->n_entries);
+	f->n_patches = le32_to_cpu(h->n_patches);
+	f->entries = ukfw_table(data, size, le32_to_cpu(h->entries_off), f->n_entries);
+	if (f->n_patches)
+		f->patches = ukfw_table(data, size, le32_to_cpu(h->patches_off), f->n_patches);
+	if (!f->entries || (f->n_patches && !f->patches))
+		return -EINVAL;
+	for (u32 i = 0; i < f->n_entries; i++) {
+		if (f->entries[i][0] > 4 || f->entries[i][1] >= isize ||
+		    !IS_ALIGNED(f->entries[i][1], 8))
+			return -EINVAL;
+		entry0 |= !f->entries[i][0];
+	}
+	for (u32 i = 0; i < f->n_patches; i++) {
+		if (f->patches[i][0] > isize - 8 || !IS_ALIGNED(f->patches[i][0], 8) ||
+		    f->patches[i][1] < UKFW_TA3DCTL || f->patches[i][1] > UKFW_PDTABLE)
+			return -EINVAL;
+	}
+	if (!entry0)
+		return -EINVAL;
+
+	f->image_size = isize;
+	f->image = kmemdup(data + hsize, isize, GFP_KERNEL);
+	return f->image ? 0 : -ENOMEM;
+}
+
+static int ukfw_load(struct pvrsgx_device *sdev, const struct firmware *fw)
+{
+	const struct ukfw_header *h = (const void *)fw->data;
+	struct pvrsgx_fw *f = &sdev->fw;
+	const char *why = "";
+	u32 iface;
+	int err;
+
+	err = ukfw_parse(fw->data, fw->size, f, &why);
+	if (err == -EINVAL)
+		drm_err(&sdev->base, "%s: %s\n", f->name, why);
+	if (err)
+		return err;
+	/*
+	 * Secure mode keeps the firmware out of every client directory, which a
+	 * single-context core can't do: there the image must not claim it.
+	 */
+	if (pvrsgx_core_single_context(sdev->family)) {
+		if ((f->features & (PVRSGX_UKFW_SECURE | PVRSGX_UKFW_SINGLE_CONTEXT)) !=
+		    PVRSGX_UKFW_SINGLE_CONTEXT) {
+			drm_err(&sdev->base, "%s: not an image for a single-context core\n",
+				f->name);
+			return -EINVAL;
+		}
+	} else if (f->features & PVRSGX_UKFW_SINGLE_CONTEXT) {
+		drm_err(&sdev->base, "%s: an image for a single-context core\n", f->name);
+		return -EINVAL;
+	} else if (!(f->features & PVRSGX_UKFW_SECURE)) {
+		drm_err(&sdev->base, "%s: not a secure-mode image; clients would reach the firmware\n",
+			f->name);
+		return -EINVAL;
+	}
+	iface = le32_to_cpu(h->interface);
+	if (h->core_rev && le32_to_cpu(h->core_rev) != sdev->core_rev)
+		drm_warn(&sdev->base, "%s: validated on core revision 0x%x, this is 0x%x\n",
+			 f->name, le32_to_cpu(h->core_rev), sdev->core_rev);
+	drm_info(&sdev->base, "%s: open microkernel %.32s, interface %u.%u, features 0x%x\n",
+		 f->name, h->build_id, iface >> 16, iface & 0xffff, f->features);
+	return 0;
+}
+
+/*
+ * Single-context cores: the microkernel loads list 0 from this table, by the
+ * slot a HW context names, so no directory address it loads is GPU-writable.
+ * Read-only on the GPU, between the code arena and the kernel code objects.
+ * Every slot no VM holds names the firmware's directory.
+ */
+#define PDTABLE_OFFSET	0xf000
+#define PDTABLE_SLOTS	(PAGE_SIZE / sizeof(u32))
+
+/* The PD table sits past the code arena, inside the kernel code heap's first 64 KiB. */
+static_assert(PDTABLE_OFFSET >= CODE_ARENA_SIZE);
+static_assert(SZ_64K - PAGE_SIZE >= PDTABLE_OFFSET);
+
+static int pdtable_init(struct pvrsgx_device *sdev)
+{
+	struct pvrsgx_fw *f = &sdev->fw;
+	int err;
+
+	if (!pvrsgx_core_single_context(sdev->family))
+		return 0;
+	err = pvrsgx_kmem_alloc(sdev, &f->pdtable,
+				pvrsgx_heaps(sdev)->kernel_code.base + PDTABLE_OFFSET, PAGE_SIZE,
+				SGX_MMU_PTE_READONLY, 0);
+	if (err)
+		return err;
+	memset32(f->pdtable.cpu, pvrsgx_mmu_pd_addr(sdev->kernel_mmu), PDTABLE_SLOTS);
+	return 0;
+}
+
+/* A slot naming directory pd, for a new VM; its HW contexts carry the slot. */
+int pvrsgx_fw_pd_slot_get(struct pvrsgx_device *sdev, u32 pd)
+{
+	int slot = ida_alloc_range(&sdev->fw.pd_slots, 1, PDTABLE_SLOTS - 1, GFP_KERNEL);
+
+	if (slot < 0)
+		return slot;
+	WRITE_ONCE(((u32 *)sdev->fw.pdtable.cpu)[slot], pd);
+	/* In memory before any command names the slot. */
+	wmb();
+	return slot;
+}
+
+/* The VM is gone: its slot names the firmware's directory again. */
+void pvrsgx_fw_pd_slot_put(struct pvrsgx_device *sdev, int slot)
+{
+	WRITE_ONCE(((u32 *)sdev->fw.pdtable.cpu)[slot], pvrsgx_mmu_pd_addr(sdev->kernel_mmu));
+	/* The slot names the firmware's directory before another VM can take it. */
+	wmb();
+	ida_free(&sdev->fw.pd_slots, slot);
+}
+
+/* The arenas and the built image, once the file is loaded. */
+static int fw_arenas(struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_heap_layout *h = pvrsgx_heaps(sdev);
+	struct pvrsgx_fw *f = &sdev->fw;
+	u32 pds_base;
+	int err;
+
+	f->interface = DRM_POWERVR_SGX_FW_IF_V1;
+
+	/*
+	 * The PTE flags: data read-write and EDM-protected where the core allows
+	 * it, code read-only.
+	 */
+	err = pvrsgx_kmem_alloc(sdev, &f->data, h->kernel_data.base, DATA_ARENA_SIZE,
+				SGX_MMU_PTE_CACHECONSISTENT |
+				(sdev->family->data_no_edmprotect ? 0 : SGX_MMU_PTE_EDMPROTECT), 0);
+	if (err)
+		return err;
+	err = pvrsgx_kmem_alloc(sdev, &f->code, h->kernel_code.base, CODE_ARENA_SIZE,
+				SGX_MMU_PTE_READONLY, 0);
+	if (err)
+		return err;
+	err = pdtable_init(sdev);
+	if (err)
+		return err;
+
+	pds_base = ALIGN(f->image_size, 0x10);
+	for (int b = 0; b < PVRSGX_FW_NBUF; b++) {
+		struct pvrsgx_kmem *km = layout[b].code ? &f->code : &f->data;
+		u32 off = layout[b].offset;
+
+		if (b >= PVRSGX_FW_PDS0 && b <= PVRSGX_FW_PDS4)
+			off += pds_base;
+		f->addr[b] = km->dev_addr + off;
+		f->cpu[b] = (u8 *)km->cpu + off;
+	}
+	return fw_build(sdev, f->image);
+}
+
+int pvrsgx_fw_init(struct pvrsgx_device *sdev)
+{
+	struct pvrsgx_fw *f = &sdev->fw;
+	const struct firmware *fw;
+	int err;
+
+	ida_init(&f->pd_slots);
+	f->name = sdev->core->fw_name;
+	err = request_firmware(&fw, f->name, sdev->base.dev);
+	if (err) {
+		drm_err(&sdev->base, "no microkernel: %s (%d)\n", f->name, err);
+		return err;
+	}
+	err = ukfw_load(sdev, fw);
+	if (err == -ENOENT)
+		drm_err(&sdev->base, "%s: not a microkernel file\n", f->name);
+	release_firmware(fw);
+	if (err)
+		return err;
+	return fw_arenas(sdev);
+}
+
+void pvrsgx_fw_fini(struct pvrsgx_device *sdev)
+{
+	kfree(sdev->fw.entries);
+	kfree(sdev->fw.patches);
+	sdev->fw.entries = NULL;
+	sdev->fw.patches = NULL;
+	kfree(sdev->fw.image);
+	sdev->fw.image = NULL;
+	pvrsgx_kmem_free(sdev, &sdev->fw.pdtable);
+	pvrsgx_kmem_free(sdev, &sdev->fw.code);
+	pvrsgx_kmem_free(sdev, &sdev->fw.data);
+	ida_destroy(&sdev->fw.pd_slots);
+}
+
+/* USE_CODE_BASE's data master for the pixel windows and for the microkernel's. */
+#define USE_CODE_BASE_DM_PIXEL		1
+#define USE_CODE_BASE_DM_UKERNEL	3
+
+static u32 use_code_base(struct pvrsgx_device *sdev, u32 n)
+{
+	return sdev->cr->use_code_base0 + 4 * n;
+}
+
+/* A code base's value: the address in the core's unit, and the data master. */
+static u32 use_code_base_val(struct pvrsgx_device *sdev, u64 addr, u32 dm)
+{
+	const struct pvrsgx_core_regs *cr = sdev->cr;
+
+	return (u32)(addr >> cr->use_code_base_addr_shift) | dm << cr->use_code_base_dm_shift;
+}
+
+/*
+ * The event program's address as the event PDS base register takes it. A core
+ * without EDM_VERTEX_PDSADDR_FULL_RANGE holds 26 bits there, the 64 MiB above
+ * PDS_EXEC_BASE that PDS code may lie in: taken as an offset from that base,
+ * which is an assumption.
+ */
+static u32 event_pds_addr(struct pvrsgx_device *sdev)
+{
+	u32 a = sdev->fw.addr[PVRSGX_FW_PDS0] & ~0xfu;
+
+	if (!(sdev->family->features & PVRSGX_FEATURE_EDM_VERTEX_PDSADDR_FULL_RANGE))
+		a -= pvrsgx_heaps(sdev)->pds_pixel.base;
+	return a;
+}
+
+/* Where the event PDS registers are a guess, the address must read back before any kick. */
+static int check_event_pds(struct pvrsgx_device *sdev)
+{
+	u32 want = event_pds_addr(sdev), got;
+
+	if (!sdev->family->event_pds_guessed)
+		return 0;
+	got = sgx_rd(sdev, sdev->cr->event_pds_base);
+	if (got != want) {
+		drm_err(&sdev->base,
+			"event PDS base 0x%04x reads 0x%08x, not 0x%08x; not starting the microkernel\n",
+			sdev->cr->event_pds_base, got, want);
+		return -EIO;
+	}
+	return 0;
+}
+
+/* Init script part 2: the core's script, then the bases. */
+static int init_script_part2(struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_core_family *fam = sdev->family;
+	const struct pvrsgx_heap_layout *h = pvrsgx_heaps(sdev);
+	const struct pvrsgx_core_regs *cr = sdev->cr;
+	struct pvrsgx_fw *f = &sdev->fw;
+	int err;
+
+	for (u32 i = 0; i < fam->num_init_regs; i++)
+		sgx_wr(sdev, fam->init_regs[i].reg, fam->init_regs[i].val);
+	/*
+	 * Code bases 0-9 the pixel shader heap, 10-13 the vertex shader heap, 15
+	 * ours. A client's PDS program can name any of them, so 14 is a second
+	 * name for 0 rather than whatever it held.
+	 */
+	for (u32 i = 0; i < 10; i++)
+		sgx_wr(sdev, use_code_base(sdev, PVRSGX_USSE_PIXEL_BASE_INDEX + i),
+		       use_code_base_val(sdev, h->usse_pixel.base + (i << PVRSGX_USSE_WINDOW_LOG2),
+					 USE_CODE_BASE_DM_PIXEL));
+	sgx_wr(sdev, use_code_base(sdev, PVRSGX_USSE_SPARE_BASE_INDEX),
+	       use_code_base_val(sdev, h->usse_pixel.base, USE_CODE_BASE_DM_PIXEL));
+	for (u32 i = 0; i < 4; i++)
+		sgx_wr(sdev, use_code_base(sdev, PVRSGX_USSE_VERTEX_BASE_INDEX + i),
+		       use_code_base_val(sdev, h->usse_vertex.base + (i << PVRSGX_USSE_WINDOW_LOG2),
+					 0));
+	sgx_wr(sdev, use_code_base(sdev, PVRSGX_USSE_UKERNEL_BASE_INDEX),
+	       use_code_base_val(sdev, h->kernel_code.base, USE_CODE_BASE_DM_UKERNEL));
+	sgx_wr(sdev, cr->pds_exec_base, h->pds_pixel.base);
+	sgx_wr_opt(sdev, cr->cr_b30, 0x100);	/* unnamed; its start value */
+	sgx_wr(sdev, SGX_CR_EVENT_KICKER, f->addr[PVRSGX_FW_KICKER]);
+	sgx_wr(sdev, cr->event_pds_base, event_pds_addr(sdev));
+	sgx_wr(sdev, cr->event_pds_data0, 1);
+	sgx_wr(sdev, cr->event_pds_data1, 6);
+	/* Before the events are enabled, so a wrong guess starts nothing. */
+	err = check_event_pds(sdev);
+	if (err)
+		return err;
+	sgx_wr(sdev, cr->event_pds_enable, cr->event_pds_enable_all);
+	return 0;
+}
+
+static void deinit_script(struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_core_regs *cr = sdev->cr;
+
+	sgx_wr(sdev, cr->event_pds_enable, 0);
+	sgx_wr(sdev, cr->event_pds_base, 0);
+	sgx_wr(sdev, cr->event_pds_data0, 0);
+	sgx_wr(sdev, cr->event_pds_data1, 0);
+	sgx_wr(sdev, use_code_base(sdev, PVRSGX_USSE_UKERNEL_BASE_INDEX), 0);
+	sgx_wr(sdev, SGX_CR_EVENT_HOST_ENABLE, 0);
+	sgx_wr(sdev, SGX_CR_EVENT_HOST_ENABLE2, 0);
+	sgx_wr(sdev, SGX_CR_CLKGATECTL, 0);
+	sgx_wr_opt(sdev, cr->clkgatectl2, 0);
+}
+
+/*
+ * Start: clocks, init script part 1, reset, part 2, then the first event kick,
+ * and the microkernel's init entry sets INIT_COMPLETE.
+ */
+int pvrsgx_fw_start(struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_core_regs *cr = sdev->cr;
+	u32 *hc = fw_words(sdev, PVRSGX_FW_HOSTCTL);
+	bool first = !sdev->fw_started, was_powered = READ_ONCE(sdev->powered);
+	int err;
+
+	WRITE_ONCE(sdev->powered, true);
+
+	/*
+	 * A restart starts from a fresh image and empty arenas: the microkernel
+	 * changes both as it runs, and a module reload (which recovers) is
+	 * exactly this.
+	 */
+	if (!first) {
+		sdev->fw_gen++;
+		/*
+		 * After a hang the clocks are on and the cores may be busy. After a
+		 * clean stop (runtime suspend) they are gated and quiet already, and
+		 * a register read there can hang the SoC.
+		 */
+		if (was_powered) {
+			/* What an unload and reload (which recovers) does in between. */
+			deinit_script(sdev);
+			sgx_wr(sdev, SGX_CR_SOFT_RESET, cr->soft_reset_bif | cr->soft_reset_cores);
+			sgx_rd(sdev, SGX_CR_SOFT_RESET);
+			/* Behind gma500 bus mastering is the display's too: leave it. */
+			if (sdev->pdev)
+				pci_clear_master(sdev->pdev);
+			fsleep(10000);
+			if (sdev->pdev)
+				pci_set_master(sdev->pdev);
+		}
+		memset(sdev->fw.data.cpu, 0, sdev->fw.data.size);
+		memset(sdev->fw.code.cpu, 0, sdev->fw.code.size);
+		err = fw_build(sdev, sdev->fw.image);
+		if (err)
+			return err;
+		/*
+		 * Nothing the old run pointed at matters any more. Freeing can take
+		 * locks a fence waiter holds, so the reap work does it, not a reset.
+		 */
+		WRITE_ONCE(sdev->restarts, sdev->restarts + 1);
+		queue_work(sdev->free_wq, &sdev->reap_work);
+	}
+	memset(hc, 0, layout[PVRSGX_FW_HOSTCTL].size);
+	hc[HOSTCTL_UK_TIMER_CLOCK] = sdev->core->clock_hz / UKERNEL_TIMER_HZ;
+	hc[HOSTCTL_FLAGS] = HOSTCTL_HOST_BGBLOCK;
+	/* The power status starts clear at every start. */
+	hc[HOSTCTL_POWER_STATUS] = 0;
+	pvrsgx_ccb_reset(sdev);
+	ta3dctl_fill(sdev);
+	/* The first command invalidates the directory cache, keeping what a reap queued. */
+	sdev->fw.cache_control |= PVRSGX_CC_INVAL_BIF_PD;
+
+	/* Every unit's clock gated automatically, but the DPM's: it stays on. */
+	sgx_wr(sdev, SGX_CR_CLKGATECTL,
+	       (SGX_CLKG_ALL(cr->clkgatectl_units, SGX_CLKG_AUTO) & ~cr->clkgatectl_dpm) |
+	       field_prep(cr->clkgatectl_dpm, SGX_CLKG_ON));
+	sgx_wr_opt(sdev, cr->clkgatectl2, SGX_CLKG_ALL(cr->clkgatectl2_units, SGX_CLKG_AUTO));
+	sgx_wr(sdev, cr->event_pds_enable, 0);
+	sgx_wr(sdev, SGX_CR_EVENT_TIMER, 0);
+
+	pvrsgx_reset(sdev, true);
+
+	*(u32 *)sdev->fw.cpu[PVRSGX_FW_KICKER] = 0;
+	err = init_script_part2(sdev);
+	if (err)
+		return err;
+
+	hc[HOSTCTL_INIT_STATUS] = 0;
+	/* The image and the host control block are in memory before the kick. */
+	wmb();
+	/* The init kick; a core with one kick register (BRN_22934) takes it there. */
+	if (cr->event_kick2)
+		sgx_wr(sdev, cr->event_kick2, cr->event_kick2_now);
+	else
+		sgx_wr(sdev, SGX_CR_EVENT_KICK, SGX_CR_EVENT_KICK_NOW);
+
+	err = pvrsgx_poll(sdev, &hc[HOSTCTL_INIT_STATUS], HOSTCTL_INIT_COMPLETE,
+			  HOSTCTL_INIT_COMPLETE, 500000);
+	if (err) {
+		drm_err(&sdev->base,
+			"microkernel did not start (init status 0x%x, event status 0x%x; BIF int 0x%x fault 0x%x req 0x%x)\n",
+			hc[HOSTCTL_INIT_STATUS], sgx_rd(sdev, SGX_CR_EVENT_STATUS),
+			sgx_rd(sdev, SGX_CR_BIF_INT_STAT), sgx_rd(sdev, SGX_CR_BIF_FAULT),
+			sgx_rd(sdev, SGX_CR_BIF_MEM_REQ_STAT));
+		return err;
+	}
+	sdev->fw_running = true;
+	sdev->fw_started = true;
+	if (first || sdev->reset_count != sdev->reset_logged) {
+		drm_info(&sdev->base, "microkernel started%s\n", first ? "" : " again");
+		sdev->reset_logged = sdev->reset_count;
+	}
+	return 0;
+}
+
+/*
+ * Every unit and the BIF held in reset, at once and without the microkernel:
+ * no GPU memory access after this returns. For a hang, before the jobs' memory
+ * can go, and when the device is lost.
+ */
+void pvrsgx_fw_halt(struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_core_regs *cr = sdev->cr;
+
+	sgx_wr(sdev, SGX_CR_SOFT_RESET, cr->soft_reset_bif | cr->soft_reset_cores);
+	sgx_rd(sdev, SGX_CR_SOFT_RESET);
+	if (sdev->pdev)
+		pci_clear_master(sdev->pdev);
+}
+
+/* Ask the microkernel to power off, then hold the cores in reset. */
+void pvrsgx_fw_stop(struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_core_regs *cr = sdev->cr;
+	u32 *hc = fw_words(sdev, PVRSGX_FW_HOSTCTL);
+	u32 data[6] = { 0, POWER_OFF };
+
+	if (sdev->fw_running) {
+		if (pvrsgx_ccb_send(sdev, PVRSGX_KCMD_POWER, data, NULL) ||
+		    pvrsgx_poll(sdev, &hc[HOSTCTL_POWER_STATUS], HOSTCTL_POWMAN_POWEROFF_COMPLETE,
+				HOSTCTL_POWMAN_POWEROFF_COMPLETE, 500000))
+			drm_warn(&sdev->base, "microkernel did not power off\n");
+		/* Wait for the clocks to gate; the DPM's stays on. */
+		for (int i = 0; i < 500; i++) {
+			if (!(sgx_rd(sdev, cr->clkgatestatus) & ~cr->clkgatestatus_dpm))
+				break;
+			usleep_range(100, 200);
+		}
+		WRITE_ONCE(sdev->powered, false);
+		synchronize_irq(sdev->irq);
+		deinit_script(sdev);
+		sdev->fw_running = false;
+		sgx_wr(sdev, SGX_CR_SOFT_RESET, cr->soft_reset_bif | cr->soft_reset_cores);
+		sgx_rd(sdev, SGX_CR_SOFT_RESET);
+	} else if (READ_ONCE(sdev->powered)) {
+		/* A failed start: the clocks are on, the microkernel isn't. */
+		WRITE_ONCE(sdev->powered, false);
+		synchronize_irq(sdev->irq);
+		sgx_wr(sdev, SGX_CR_SOFT_RESET, cr->soft_reset_bif | cr->soft_reset_cores);
+		sgx_rd(sdev, SGX_CR_SOFT_RESET);
+	}
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.h
new file mode 100644
index 000000000..92dfa0898
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.h
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_FW_H
+#define PVRSGX_FW_H
+
+#include <linux/bits.h>
+#include <linux/types.h>
+
+struct pvrsgx_device;
+struct seq_buf;
+
+/* Fixed device-virtual layout. */
+#define PVRSGX_HEAP_3D_PARAMS		0x10000000u
+#define PVRSGX_HEAP_TA_DATA		0x20000000u
+#define PVRSGX_HEAP_SYNC		0x2d000000u
+#define PVRSGX_HEAP_PDS_PIXEL		0x2e000000u
+#define PVRSGX_HEAP_KERNEL_CODE		0x30000000u
+#define PVRSGX_HEAP_PDS_VERTEX		0x30400000u
+#define PVRSGX_HEAP_KERNEL_DATA		0x32000000u
+#define PVRSGX_HEAP_USSE_PIXEL		0x37000000u
+#define PVRSGX_HEAP_USSE_PIXEL_SIZE	0x00500000u
+#define PVRSGX_HEAP_USSE_VERTEX		0x3c000000u
+#define PVRSGX_HEAP_USSE_VERTEX_SIZE	0x00200000u
+#define PVRSGX_HEAP_GENERAL		0x3e000000u
+#define PVRSGX_HEAP_TEXTURE		0xf0000000u
+
+/* Each USE code base register covers a 512 KiB window. */
+#define PVRSGX_USSE_WINDOW_LOG2		19
+#define PVRSGX_USSE_PIXEL_BASE_INDEX	0
+#define PVRSGX_USSE_VERTEX_BASE_INDEX	10
+#define PVRSGX_USSE_SPARE_BASE_INDEX	14
+#define PVRSGX_USSE_UKERNEL_BASE_INDEX	15
+
+/* Host control block words. */
+#define HOSTCTL_INIT_STATUS		0
+#define HOSTCTL_POWER_STATUS		1
+#define HOSTCTL_UK_LOCKUPS		3
+#define HOSTCTL_UK_TIMER_CLOCK		6
+#define HOSTCTL_INTERRUPT_FLAGS		8
+#define HOSTCTL_INTERRUPT_CLEAR		9
+#define HOSTCTL_ASSERT_FAIL		14
+
+#define HOSTCTL_FLAGS			15
+
+/* HOSTCTL_FLAGS: the host writes the background object's PDS words. */
+#define HOSTCTL_HOST_BGBLOCK		BIT(0)
+
+/* Feature bits of the microkernel file's header. */
+#define PVRSGX_UKFW_CLEANUP_V2		BIT(0)	/* accepted, unused */
+#define PVRSGX_UKFW_SECURE		BIT(1)
+/* It counts finished jobs in pvrsgx_fw_ta3d_ctl.done_seq, then interrupts. */
+#define PVRSGX_UKFW_DONE_SEQ		BIT(2)
+/*
+ * Runs on list 0 with the jobs, directories from the PD table, the firmware
+ * directory back on list 0 before any completion (single-context cores).
+ */
+#define PVRSGX_UKFW_SINGLE_CONTEXT	BIT(3)
+
+#define HOSTCTL_INIT_COMPLETE		BIT(0)
+#define HOSTCTL_POWMAN_POWEROFF_COMPLETE BIT(3)
+#define HOSTCTL_INTERRUPT_HWR		BIT(0)
+#define HOSTCTL_INTERRUPT_ACTIVE_POWER	BIT(1)
+
+int pvrsgx_fw_init(struct pvrsgx_device *sdev);
+int pvrsgx_fw_pd_slot_get(struct pvrsgx_device *sdev, u32 pd);
+void pvrsgx_fw_pd_slot_put(struct pvrsgx_device *sdev, int slot);
+void pvrsgx_fw_fini(struct pvrsgx_device *sdev);
+int pvrsgx_fw_start(struct pvrsgx_device *sdev);
+void pvrsgx_fw_stop(struct pvrsgx_device *sdev);
+void pvrsgx_fw_halt(struct pvrsgx_device *sdev);
+u32 pvrsgx_fw_hostctl(struct pvrsgx_device *sdev, unsigned int word);
+void pvrsgx_fw_hostctl_set(struct pvrsgx_device *sdev, unsigned int word, u32 v);
+void pvrsgx_fw_dump(struct seq_buf *sb, struct pvrsgx_device *sdev, unsigned int b,
+		    const char *name);
+
+#endif /* PVRSGX_FW_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_fwif.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_fwif.h
new file mode 100644
index 000000000..9fa7b64c0
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_fwif.h
@@ -0,0 +1,519 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_FWIF_H
+#define PVRSGX_FWIF_H
+
+#include <linux/bits.h>
+#include <linux/build_bug.h>
+#include <linux/stddef.h>
+#include <linux/types.h>
+
+/*
+ * Structures the host writes and the microkernel reads, in the microkernel's
+ * layout. The SGX and its x86 hosts are both little-endian, so the fields are
+ * plain u32. Fields the driver leaves zero are padding here.
+ */
+
+/**
+ * struct pvrsgx_fw_ta3d_ctl - The microkernel's control block
+ *
+ * The host fills the addresses;
+ * the rest is the microkernel's live state, zero at every start.
+ */
+struct pvrsgx_fw_ta3d_ctl {
+	/** @self: Device address of this block. */
+	u32 self;
+	/** @host_ctl: Device address of the host control block. */
+	u32 host_ctl;
+	/** @ccb_ctl: Device address of the kernel CCB's write and read offsets. */
+	u32 ccb_ctl;
+	/* private: */
+	u32 pad_0c[19];
+	/* public: */
+	/** @ccb: Device address of the kernel CCB. */
+	u32 ccb;
+	/** @kernel_pd: Physical address of the firmware's page directory. */
+	u32 kernel_pd;
+	/* private: the microkernel's own state, to the block's end (word 0x3f) */
+	u32 uk_60[15];
+	/* @done_seq: Jobs finished since the start, with PVRSGX_UKFW_DONE_SEQ. */
+	u32 done_seq;
+	u32 uk_a0[24];
+};
+
+static_assert(offsetof(struct pvrsgx_fw_ta3d_ctl, ccb) == 0x58);
+static_assert(offsetof(struct pvrsgx_fw_ta3d_ctl, done_seq) == 0x9c);
+static_assert(sizeof(struct pvrsgx_fw_ta3d_ctl) == 0x100);
+
+/* struct pvrsgx_fw_ta_cmd.ta_flags */
+#define PVRSGX_TA_FIRST_KICK		BIT(0)
+#define PVRSGX_TA_LAST_KICK		BIT(1)
+#define PVRSGX_TA_FIRST_RENDER		BIT(4)	/* the RT dataset's first render */
+#define PVRSGX_TA_FLAGS_BASE		0x60	/* set on every kick */
+
+/* struct pvrsgx_fw_ta_cmd.ta_flags2 */
+#define PVRSGX_TA2_RENDER		BIT(22)	/* this kick starts the 3D */
+
+/* struct pvrsgx_fw_ta_cmd.flags_3d */
+#define PVRSGX_3D_RENDER		BIT(10)
+#define PVRSGX_3D_FRAME_SHIFT		14
+
+/**
+ * struct pvrsgx_fw_ta_cmd - One TA kick
+ *
+ * The header and tail come from the client's streams and the kernel's own
+ * structures; the shared part carries the synchronization.
+ */
+struct pvrsgx_fw_ta_cmd {
+	/** @size: The command's size in bytes. */
+	u32 size;
+	/** @ta_flags: PVRSGX_TA_*. */
+	u32 ta_flags;
+	/** @bg_pds: Background object PDS words: secondary, middle, primary. */
+	u32 bg_pds[3];
+	/** @ta_ctl: Two constant control words. */
+	u32 ta_ctl[2];
+	/** @flags_3d: 3D flags, on the scene's last kick. */
+	u32 flags_3d;
+	/** @frame: The RT dataset's frame counter. */
+	u32 frame;
+	/** @pb_desc: Device address of the parameter buffer descriptor. */
+	u32 pb_desc;
+	/* private: */
+	u32 pad_28[4];
+	/* public: */
+	/** @render_details: Device address of the render details. */
+	u32 render_details;
+	/** @rt_deps: Device address of the render-target dependency block. */
+	u32 rt_deps;
+	/** @rt_data: Device address of the RT data. */
+	u32 rt_data;
+	/** @rt_data_entry: Device address of this render's RT data entry. */
+	u32 rt_data_entry;
+	/* private: */
+	u32 pad_48[4];
+	/* public: */
+
+	/** @ready: Set when the command may run. */
+	u32 ready;
+	/* private: */
+	u32 pad_5c[6];
+	/* public: */
+	/** @sync3d_write_pending: The 3D sync's write-ops-pending value. */
+	u32 sync3d_write_pending;
+	/** @sync3d_write_complete: Its write-ops-complete address. */
+	u32 sync3d_write_complete;
+	/** @sync3d_read_pending: The 3D sync's read-ops-pending value. */
+	u32 sync3d_read_pending;
+	/** @sync3d_read_complete: Its read-ops-complete address. */
+	u32 sync3d_read_complete;
+	/* private: */
+	u32 pad_84[109];
+	/* public: */
+
+	/** @te_aa: Tiling engine anti-aliasing word. */
+	u32 te_aa;
+	/** @te_mtile: Macrotile boundaries in X and Y. */
+	u32 te_mtile[2];
+	/** @te_screen: Last tile in X (bits 0-11) and Y (bits 12 on). */
+	u32 te_screen;
+	/** @mt_tiles: Tiles per macrotile. */
+	u32 mt_tiles;
+	/** @ta_flags2: PVRSGX_TA2_* and the client's kick flags. */
+	u32 ta_flags2;
+	/** @vdm_base: Device address of the kick's VDM control stream. */
+	u32 vdm_base;
+	/** @vdm_ctl: Constant word next to it. */
+	u32 vdm_ctl;
+	/** @region_headers: This render's region header array. */
+	u32 region_headers;
+	/** @tail_pointers: The tail pointer array. */
+	u32 tail_pointers;
+	/** @eps: Two floats, 1e-6. */
+	u32 eps[2];
+	/** @screen_size: (height - 1) << 16 | (width - 1). */
+	u32 screen_size;
+	/* private: */
+	u32 pad_26c[2];
+	/* public: */
+	/** @params_base: Base of the 3D parameters heap the PB is in. */
+	u32 params_base;
+	/* private: */
+	u32 pad_278[11];
+	/* public: */
+	/** @rt_flags: RT data flags. */
+	u32 rt_flags;
+	/* private: */
+	u32 pad_2a8[6];
+};
+
+static_assert(offsetof(struct pvrsgx_fw_ta_cmd, ready) == 0x58);
+static_assert(offsetof(struct pvrsgx_fw_ta_cmd, te_aa) == 0x238);
+static_assert(offsetof(struct pvrsgx_fw_ta_cmd, params_base) == 0x274);
+static_assert(offsetof(struct pvrsgx_fw_ta_cmd, rt_flags) == 0x2a4);
+static_assert(sizeof(struct pvrsgx_fw_ta_cmd) == 0x2c0);
+
+/**
+ * struct pvrsgx_fw_rt_deps - A render target's dependency block
+ *
+ * What the 3D signals when it finishes: one destination sync, whose
+ * write-ops-complete the microkernel sets to the pending value plus one.
+ */
+struct pvrsgx_fw_rt_deps {
+	/** @access: Device address of the access word. */
+	u32 access;
+	/** @num_dst: Destination syncs, 1. */
+	u32 num_dst;
+	/** @read_pending: The destination's read-ops-pending value. */
+	u32 read_pending;
+	/** @read_complete: Its read-ops-complete address. */
+	u32 read_complete;
+	/** @write_pending: Its write-ops-pending value. */
+	u32 write_pending;
+	/** @write_complete: Its write-ops-complete address. */
+	u32 write_complete;
+	/* private: */
+	u32 pad_18[4];
+};
+
+static_assert(sizeof(struct pvrsgx_fw_rt_deps) == 0x28);
+
+/**
+ * struct pvrsgx_fw_rt_entry - One of an RT dataset's two render slots
+ *
+ * Frames alternate between the two.
+ */
+struct pvrsgx_fw_rt_entry {
+	/** @hw_render_ctx: Device address of the HW render context. */
+	u32 hw_render_ctx;
+	/** @render_details: The render in flight; the microkernel's. */
+	u32 render_details;
+	/** @mt_units: Tiles per macrotile >> 4. */
+	u32 mt_units;
+	/** @mt_bytes: Tiles per macrotile * 0xc. */
+	u32 mt_bytes;
+	/** @bg_block: Device address of the background object block. */
+	u32 bg_block;
+	/** @ta_data: The 0x88-, 0x40- and 0x50-byte TA data blocks. */
+	u32 ta_data[3];
+	/** @tail_pointers: The tail pointer array. */
+	u32 tail_pointers;
+	/** @tail_pointers_size: Its size in bytes. */
+	u32 tail_pointers_size;
+	/** @num_rts: Render targets, 1. */
+	u32 num_rts;
+	/* private: */
+	u32 pad_2c;
+	/* public: */
+	/** @entry_word: Device address of the slot's status word. */
+	u32 entry_word;
+	/** @unk_34: 0x10. */
+	u32 unk_34;
+};
+
+static_assert(sizeof(struct pvrsgx_fw_rt_entry) == 0x38);
+
+/**
+ * struct pvrsgx_fw_rt_data - An RT dataset's RT data
+ */
+struct pvrsgx_fw_rt_data {
+	/** @rt_word: Device address of the dataset's status word. */
+	u32 rt_word;
+	/* private: */
+	u32 pad_04;
+	/* public: */
+	/** @padded_tiles_x: Tiles in X, padded to whole macrotiles. */
+	u32 padded_tiles_x;
+	/* private: */
+	u32 pad_0c[2];
+	/* public: */
+	/** @num_entries: Render slots, 2. */
+	u32 num_entries;
+	/** @entry: The render slots. */
+	struct pvrsgx_fw_rt_entry entry[2];
+};
+
+static_assert(sizeof(struct pvrsgx_fw_rt_data) == 0x88);
+
+/**
+ * struct pvrsgx_fw_pb_desc - Parameter buffer descriptor
+ *
+ * Page numbers count from the 3D parameters heap.
+ */
+struct pvrsgx_fw_pb_desc {
+	/* private: */
+	u32 pad_00;
+	/* public: */
+	/** @pages: Pages in the PB. */
+	u32 pages;
+	/** @free_head: Free list head: last page << 16 | first page. */
+	u32 free_head;
+	/** @free_tail: (last page - 1) << 16. */
+	u32 free_tail;
+	/** @free_list: Device address of the free list. */
+	u32 free_list;
+	/* private: */
+	u32 pad_14;
+	/* public: */
+	/** @free_head_copy: The free list head again. */
+	u32 free_head_copy;
+	/** @free_tail_copy: The tail word again. */
+	u32 free_tail_copy;
+	/* private: */
+	u32 pad_20[2];
+	/* public: */
+	/** @ta_threshold: Pages the TA may use: pages + 0x10. */
+	u32 ta_threshold;
+	/** @zls_threshold: Pages for depth/stencil load/store: the request. */
+	u32 zls_threshold;
+	/** @global_threshold: The request. */
+	u32 global_threshold;
+	/** @pds_threshold: min(ZLS, TA) - 0x100. */
+	u32 pds_threshold;
+	/** @blocks: Device address of the PB block list. */
+	u32 blocks;
+};
+
+static_assert(sizeof(struct pvrsgx_fw_pb_desc) == 0x3c);
+
+/**
+ * struct pvrsgx_fw_render_details - Per-dataset render details
+ *
+ * Fields named unk_* hold values a render needs whose meaning isn't known.
+ */
+struct pvrsgx_fw_render_details {
+	/** @req_base_3d: 3D request base: the PB's 1 MiB-aligned base. */
+	u32 req_base_3d;
+	/** @zls_base: Depth/stencil request base, 1 MiB aligned. */
+	u32 zls_base;
+	/** @unk_08: 0x80. */
+	u32 unk_08;
+	/** @unk_0c: 0xf0000. */
+	u32 unk_0c;
+	/** @rt_ctl: Bit 31. */
+	u32 rt_ctl;
+	/** @depth_bias_units: glPolygonOffset units. */
+	u32 depth_bias_units;
+	/** @depth_bias_factor: glPolygonOffset factor, a float. */
+	u32 depth_bias_factor;
+	/* private: */
+	u32 pad_1c;
+	/* public: */
+	/** @region_headers: This render's region header array. */
+	u32 region_headers;
+	/* private: */
+	u32 pad_24;
+	/* public: */
+	/** @region_headers_next: The next render's. */
+	u32 region_headers_next;
+	/** @zls_ctl: Depth/stencil load and store control. */
+	u32 zls_ctl;
+	/** @depth: Depth buffer address - @zls_base, twice. */
+	u32 depth[2];
+	/** @stencil: Bit 31 | stencil buffer address - @zls_base, twice. */
+	u32 stencil[2];
+	/** @bg_depth: Depth every tile starts from, a float. */
+	u32 bg_depth;
+	/** @unk_44: 0x300. */
+	u32 unk_44;
+	/** @bg_object: 0x2000000 | (background block - PB base) >> 4. */
+	u32 bg_object;
+	/* private: */
+	u32 pad_4c;
+	/* public: */
+	/** @rt_flags: RT flags, 0x88. */
+	u32 rt_flags;
+	/* private: */
+	u32 pad_54;
+	/* public: */
+	/** @unk_58: 0x30. */
+	u32 unk_58;
+	/** @pixel_event_pds: End-of-tile PDS program. */
+	u32 pixel_event_pds;
+	/** @pixel_event_rows: Its data rows. */
+	u32 pixel_event_rows;
+	/** @unk_64: 1. */
+	u32 unk_64;
+	/** @unk_68: 1. */
+	u32 unk_68;
+	/* private: */
+	u32 pad_6c[24];
+	/* public: */
+	/** @unk_cc: 0x100. */
+	u32 unk_cc;
+	/** @sync: Device address of the render-details sync word. */
+	u32 sync;
+	/* private: */
+	u32 pad_d4;
+	/* public: */
+	/** @rt_data: Device address of the RT data. */
+	u32 rt_data;
+	/** @rt_data_entry: This render's RT data entry. */
+	u32 rt_data_entry;
+	/* private: */
+	u32 pad_e0[9];
+	/* public: */
+	/** @unk_104: 1. */
+	u32 unk_104;
+	/* private: */
+	u32 pad_108[2];
+	/* public: */
+	/** @word8: Device address of an 8-byte kernel word. */
+	u32 word8;
+	/** @bg_pds_word: Device address of the background object's primary PDS word. */
+	u32 bg_pds_word;
+	/* private: */
+	u32 pad_118[9];
+	/* public: */
+	/** @unk_13c: 0x7c00000. */
+	u32 unk_13c;
+	/* private: */
+	u32 pad_140[43];
+};
+
+static_assert(offsetof(struct pvrsgx_fw_render_details, unk_cc) == 0xcc);
+static_assert(offsetof(struct pvrsgx_fw_render_details, unk_104) == 0x104);
+static_assert(offsetof(struct pvrsgx_fw_render_details, unk_13c) == 0x13c);
+static_assert(sizeof(struct pvrsgx_fw_render_details) == 0x1ec);
+
+/* struct pvrsgx_fw_hw_render_ctx.flags */
+#define PVRSGX_HW_RENDER_CTX_VALID	BIT(0)
+#define PVRSGX_HW_RENDER_CTX_OWN_PB	BIT(5)
+
+/**
+ * struct pvrsgx_fw_hw_render_ctx - A render context, as the microkernel sees it
+ */
+struct pvrsgx_fw_hw_render_ctx {
+	/** @flags: PVRSGX_HW_RENDER_CTX_*. */
+	u32 flags;
+	/* private: */
+	u32 pad_04;
+	/* public: */
+	/** @page_dir: Physical address of the context's page directory. */
+	u32 page_dir;
+	/** @ccb: Device address of the context's command ring. */
+	u32 ccb;
+	/** @ccb_ctl: Device address of its write and read offsets. */
+	u32 ccb_ctl;
+	/* private: */
+	u32 pad_14;
+	/* public: */
+	/** @pb_desc: Device address of the PB descriptor. */
+	u32 pb_desc;
+	/** @scratch: Device address of the microkernel's per-context scratch block. */
+	u32 scratch;
+	/* private: */
+	u32 pad_20[8];
+	/* public: */
+	/** @pid: The creating process. */
+	u32 pid;
+};
+
+static_assert(sizeof(struct pvrsgx_fw_hw_render_ctx) == 0x44);
+
+/**
+ * struct pvrsgx_fw_hw_transfer_ctx - A transfer context, as the microkernel sees it
+ */
+struct pvrsgx_fw_hw_transfer_ctx {
+	/** @flags: 1. */
+	u32 flags;
+	/* private: */
+	u32 pad_04;
+	/* public: */
+	/** @page_dir: Physical address of the context's page directory. */
+	u32 page_dir;
+	/** @ccb: Device address of the context's command ring. */
+	u32 ccb;
+	/** @ccb_ctl: Device address of its write and read offsets. */
+	u32 ccb_ctl;
+	/* private: */
+	u32 pad_14[37];
+	/* public: */
+	/** @pid: The creating process. */
+	u32 pid;
+	/* private: */
+	u32 pad_ac[2];
+};
+
+static_assert(offsetof(struct pvrsgx_fw_hw_transfer_ctx, pid) == 0xa8);
+static_assert(sizeof(struct pvrsgx_fw_hw_transfer_ctx) == 0xb4);
+
+/* struct pvrsgx_fw_transfer_cmd.flags */
+#define PVRSGX_TRANSFER_DST_TWIDDLED	BIT(3)
+
+/**
+ * struct pvrsgx_fw_transfer_cmd - One transfer: a 3D render of a background object
+ *
+ * The first words are 3D registers in the
+ * render details' encoding; the shared part carries the destination sync.
+ */
+struct pvrsgx_fw_transfer_cmd {
+	/** @bg_object: 0x2000000 | (background block - @req_base_3d) >> 4. */
+	u32 bg_object;
+	/** @unk_04: 0x200. */
+	u32 unk_04;
+	/** @bg_depth: Depth every tile starts from, a float. */
+	u32 bg_depth;
+	/** @unk_0c: 3. */
+	u32 unk_0c;
+	/* private: */
+	u32 pad_10;
+	/* public: */
+	/** @unk_14: 0x100. */
+	u32 unk_14;
+	/** @req_base_3d: 3D request base, 1 MiB aligned. */
+	u32 req_base_3d;
+	/** @rt_ctl: Bit 31. */
+	u32 rt_ctl;
+	/** @rt_flags: 0x88. */
+	u32 rt_flags;
+	/** @pixel_event_pds: End-of-tile PDS program. */
+	u32 pixel_event_pds;
+	/** @pixel_event_rows: Its data rows. */
+	u32 pixel_event_rows;
+	/** @unk_2c: 1. */
+	u32 unk_2c;
+	/** @first_tile: First tile, x << 16 | y. */
+	u32 first_tile;
+	/** @last_tile: Last tile, x << 16 | y. */
+	u32 last_tile;
+	/** @unk_38: 0x80. */
+	u32 unk_38;
+	/** @pds_exec_base: PDS execution base: the pixel PDS heap. */
+	u32 pds_exec_base;
+	/* private: */
+	u32 pad_40[25];
+	/* public: */
+	/** @size: The command's size in bytes. */
+	u32 size;
+	/** @flags: PVRSGX_TRANSFER_*. */
+	u32 flags;
+	/* private: */
+	u32 pad_ac[33];
+	/* public: */
+	/** @num_dst: Destination syncs, 1. */
+	u32 num_dst;
+	/** @dst_read_pending: The destination sync's read-ops-pending value. */
+	u32 dst_read_pending;
+	/** @dst_read_complete: Its read-ops-complete address. */
+	u32 dst_read_complete;
+	/** @dst_write_pending: Its write-ops-pending value. */
+	u32 dst_write_pending;
+	/** @dst_write_complete: Its write-ops-complete address. */
+	u32 dst_write_complete;
+	/* private: */
+	u32 pad_144[20];
+	/* public: */
+	/** @unk_194: 0x7c00000. */
+	u32 unk_194;
+	/* private: */
+	u32 pad_198[10];
+};
+
+static_assert(offsetof(struct pvrsgx_fw_transfer_cmd, size) == 0xa4);
+static_assert(offsetof(struct pvrsgx_fw_transfer_cmd, num_dst) == 0x130);
+static_assert(offsetof(struct pvrsgx_fw_transfer_cmd, unk_194) == 0x194);
+static_assert(sizeof(struct pvrsgx_fw_transfer_cmd) == 0x1c0);
+
+#endif /* PVRSGX_FWIF_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_pds.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_pds.c
new file mode 100644
index 000000000..14afb4814
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_pds.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The PDS programs that enter the microkernel, assembled at load.
+ */
+
+#include "pvrsgx_fw.h"
+#include "pvrsgx_pds.h"
+
+/*
+ * DOUTU word 0 for microkernel code at device address a: code base 15 plus the
+ * 512 KiB segment, then the offset in the segment in 8-byte instructions, << 4.
+ * Bit 27 is set on every microkernel entry.
+ */
+u32 pvrsgx_doutu(u32 kcode, u32 a)
+{
+	u32 off = a - kcode, seg = off >> 19;
+
+	return ((seg + 15) & 15) | 0x8000000 | (((off - (seg << 19)) * 2) & 0x3ffff0);
+}
+
+void pvrsgx_pds_ds(struct pvrsgx_pds *p, unsigned int r, u32 v)
+{
+	unsigned int k = r & 0x3f;
+
+	p->w[(k >> 1) * 4 + (r >= 0x40 ? 2 : 0) + (k & 1)] = v;
+}
+
+static void pds_emit(struct pvrsgx_pds *p, u32 i)
+{
+	p->w[p->data + p->pc++] = i;
+}
+
+static unsigned int ds0w(unsigned int r)
+{
+	return r >= 0x80 ? r - 0x40 : r;
+}
+
+/* MOVS: four words from one DS0 pair and one DS1 pair to a DOUT destination. */
+void pvrsgx_pds_movs(struct pvrsgx_pds *p, unsigned int dest, unsigned int a,
+		     unsigned int b, unsigned int c, unsigned int d)
+{
+	unsigned int src[4] = { a, b, c, d }, r1 = 0, r2 = 0, sel = 0;
+
+	for (int i = 0; i < 4; i++) {
+		unsigned int s;
+
+		if (src[i] >= 0x40 && src[i] < 0x80) {
+			r2 = (src[i] - 0x40) >> 1;
+			s = 2 + (src[i] & 1);
+		} else {
+			r1 = ds0w(src[i]) >> 1;
+			s = ds0w(src[i]) & 1;
+		}
+		sel |= s << (11 - 2 * i);
+	}
+	pds_emit(p, (u32)PDS_CC_ALWAYS << 24 | r1 << 18 | r2 << 13 | sel | dest);
+}
+
+void pvrsgx_pds_halt(struct pvrsgx_pds *p)
+{
+	pds_emit(p, 0xaf000000u);
+}
+
+struct pvrsgx_pds pvrsgx_pds_prog(struct pvrsgx_device *sdev, enum pvrsgx_fw_buf b,
+				  unsigned int data)
+{
+	return (struct pvrsgx_pds){ sdev->fw.cpu[b], data, 0 };
+}
+
+/* Start the microkernel at offset e: DOUTU with word 1 = 2. */
+void pvrsgx_pds_entry(struct pvrsgx_device *sdev, enum pvrsgx_fw_buf b, u32 e)
+{
+	struct pvrsgx_pds p = pvrsgx_pds_prog(sdev, b, 4);
+	u32 doutu = pvrsgx_doutu(pvrsgx_heaps(sdev)->kernel_code.base,
+				 sdev->fw.addr[PVRSGX_FW_UKERNEL] + e);
+
+	if (sdev->family->features & PVRSGX_FEATURE_PDS_DATA_INTERLEAVE_2DWORDS) {
+		pvrsgx_pds_ds(&p, PDS_D0(0), doutu);
+		pvrsgx_pds_ds(&p, PDS_D0(1), 2);
+		pvrsgx_pds_movs(&p, PDS_DOUTU, PDS_D0(0), PDS_D0(1), PDS_D1(0), PDS_D0(0));
+	} else {
+		/*
+		 * Whether the data store interleaves one word at a time is unknown:
+		 * the memory row [doutu, 0, 0, 2] gives DOUTU (doutu, 2, 0)
+		 * under either layout.
+		 */
+		pvrsgx_pds_ds(&p, PDS_D0(0), doutu);
+		pvrsgx_pds_ds(&p, PDS_D0(1), 0);
+		pvrsgx_pds_ds(&p, PDS_D1(0), 0);
+		pvrsgx_pds_ds(&p, PDS_D1(1), 2);
+		pvrsgx_pds_movs(&p, PDS_DOUTU, PDS_D0(0), PDS_D1(1), PDS_D0(1), PDS_D0(0));
+	}
+	pvrsgx_pds_halt(&p);
+}
+
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_pds.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_pds.h
new file mode 100644
index 000000000..c5d1453c9
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_pds.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_PDS_H
+#define PVRSGX_PDS_H
+
+#include <linux/types.h>
+
+#include "pvrsgx_device.h"
+
+/*
+ * PDS assembly. A program is its data segment in 16-byte rows of
+ * DS0 DS0 DS1 DS1, then code. Operands are unified register numbers: DS0 words
+ * 0x00-0x3f, DS1 words 0x40-0x7f, inputs from 0x80 (DS0 words 0x40 on).
+ */
+#define PDS_D0(n)		(n)
+#define PDS_D1(n)		(0x40 + (n))
+#define PDS_CC_ALWAYS	7
+#define PDS_DOUTU		5
+
+/* A PDS program being assembled into a firmware buffer. */
+struct pvrsgx_pds {
+	u32 *w;
+	unsigned int data;	/* data segment words */
+	unsigned int pc;
+};
+
+u32 pvrsgx_doutu(u32 kcode, u32 addr);
+void pvrsgx_pds_ds(struct pvrsgx_pds *p, unsigned int r, u32 v);
+void pvrsgx_pds_movs(struct pvrsgx_pds *p, unsigned int dest, unsigned int a,
+		     unsigned int b, unsigned int c, unsigned int d);
+void pvrsgx_pds_halt(struct pvrsgx_pds *p);
+struct pvrsgx_pds pvrsgx_pds_prog(struct pvrsgx_device *sdev, enum pvrsgx_fw_buf b,
+				  unsigned int data);
+void pvrsgx_pds_entry(struct pvrsgx_device *sdev, enum pvrsgx_fw_buf b, u32 e);
+
+#endif /* PVRSGX_PDS_H */

-- 
2.50.1

[PATCH RFC 07/13] drm/powervr_sgx: Add GEM objects, VMs and dma-buf sharing

[Joseph Telaak]

- GEM shmem objects below 4 GiB, mapped write-combined. A code object
  has no CPU mapping; its contents come through UPLOAD_CODE.
- VM contexts on drm_gpuvm, one GPU page directory each, with VM_BIND
  mapping and unmapping inside the fixed heaps. An unmap invalidates the
  GPU's directory and table caches and waits for the VM's jobs before
  the pages go.
- dma-buf export and import (import on x86 only for now). The SGX
  doesn't snoop the CPU's caches, so an import is written back once when
  it is imported, through the exporter's vmap, and the byte ranges a job
  lists are flushed before it runs and, where it writes, again before
  its fence signals.
- Firmware objects a client releases wait until the GPU can no longer
  use them; a VM's mappings go when its handle and its last context are
  gone.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/powervr_sgx/pvrsgx_gem.c   | 120 +++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_gem.h   |  46 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.c  | 285 ++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.h  |  82 +++
 drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c | 203 ++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_prime.h |  26 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_vm.c    | 799 +++++++++++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_vm.h    |  69 +++
 8 files changed, 1630 insertions(+)

diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_gem.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_gem.c
new file mode 100644
index 000000000..b22b819b0
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_gem.c
@@ -0,0 +1,120 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+#include <drm/drm_gem.h>
+#include <drm/drm_prime.h>
+#include <drm/drm_vma_manager.h>
+
+#include <linux/dma-buf.h>
+#include <linux/pagemap.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_drv.h"
+#include "pvrsgx_gem.h"
+
+/* The GPU has 32-bit physical addresses and a 4 GiB address space. */
+#define PVRSGX_BO_MAX_SIZE	SZ_1G
+
+/* A code object's contents are validated: no other way to reach them. */
+static struct dma_buf *pvrsgx_gem_export(struct drm_gem_object *obj, int flags)
+{
+	if (to_pvrsgx_bo(obj)->flags & DRM_POWERVR_SGX_BO_CODE)
+		return ERR_PTR(-EPERM);
+	return drm_gem_prime_export(obj, flags);
+}
+
+/* Every CPU mapping, the dma-buf's included, needs ALLOW_CPU_ACCESS. */
+static int pvrsgx_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
+{
+	if (!drm_gem_is_imported(obj) &&
+	    !(to_pvrsgx_bo(obj)->flags & DRM_POWERVR_SGX_BO_ALLOW_CPU_ACCESS))
+		return -EINVAL;
+	return drm_gem_shmem_object_mmap(obj, vma);
+}
+
+static void pvrsgx_gem_free(struct drm_gem_object *obj)
+{
+	struct pvrsgx_bo *bo = to_pvrsgx_bo(obj);
+
+	if (drm_gem_is_imported(obj) && !iosys_map_is_null(&bo->import_map))
+		dma_buf_vunmap_unlocked(obj->import_attach->dmabuf, &bo->import_map);
+	drm_gem_shmem_object_free(obj);
+}
+
+static const struct drm_gem_object_funcs pvrsgx_gem_funcs = {
+	.free = pvrsgx_gem_free,
+	.export = pvrsgx_gem_export,
+	.print_info = drm_gem_shmem_object_print_info,
+	.pin = drm_gem_shmem_object_pin,
+	.unpin = drm_gem_shmem_object_unpin,
+	.get_sg_table = drm_gem_shmem_object_get_sg_table,
+	.vmap = drm_gem_shmem_object_vmap,
+	.vunmap = drm_gem_shmem_object_vunmap,
+	.mmap = pvrsgx_gem_mmap,
+	.vm_ops = &drm_gem_shmem_vm_ops,
+};
+
+struct drm_gem_object *pvrsgx_gem_create_object(struct drm_device *drm, size_t size)
+{
+	struct pvrsgx_bo *bo = kzalloc_obj(*bo);
+
+	if (!bo)
+		return ERR_PTR(-ENOMEM);
+	bo->base.base.funcs = &pvrsgx_gem_funcs;
+	/* The GPU doesn't snoop the CPU caches. */
+	bo->base.map_wc = true;
+	return &bo->base.base;
+}
+
+int pvrsgx_ioctl_create_bo(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_create_bo_args *args = data;
+	struct drm_gem_shmem_object *shmem;
+	int err;
+
+	if (args->_padding_c || (args->flags & ~PVRSGX_BO_FLAGS_MASK) ||
+	    !args->size || !PAGE_ALIGNED(args->size) || args->size > PVRSGX_BO_MAX_SIZE)
+		return -EINVAL;
+	if ((args->flags & DRM_POWERVR_SGX_BO_CODE) &&
+	    (args->flags & DRM_POWERVR_SGX_BO_ALLOW_CPU_ACCESS))
+		return -EINVAL;
+
+	shmem = drm_gem_shmem_create(drm, args->size);
+	if (IS_ERR(shmem))
+		return PTR_ERR(shmem);
+	to_pvrsgx_bo(&shmem->base)->flags = args->flags;
+	/* Pages the GPU can address. */
+	mapping_set_gfp_mask(shmem->base.filp->f_mapping,
+			     (GFP_HIGHUSER & ~__GFP_HIGHMEM) | __GFP_DMA32);
+
+	err = drm_gem_handle_create(file, &shmem->base, &args->handle);
+	drm_gem_object_put(&shmem->base);
+	return err;
+}
+
+int pvrsgx_ioctl_get_bo_mmap_offset(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_get_bo_mmap_offset_args *args = data;
+	struct drm_gem_object *obj;
+	int err;
+
+	if (args->_padding_4)
+		return -EINVAL;
+
+	obj = drm_gem_object_lookup(file, args->handle);
+	if (!obj)
+		return -ENOENT;
+	if (!(to_pvrsgx_bo(obj)->flags & DRM_POWERVR_SGX_BO_ALLOW_CPU_ACCESS)) {
+		err = -EINVAL;
+		goto out;
+	}
+	err = drm_gem_create_mmap_offset(obj);
+	if (!err)
+		args->offset = drm_vma_node_offset_addr(&obj->vma_node);
+out:
+	drm_gem_object_put(obj);
+	return err;
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_gem.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_gem.h
new file mode 100644
index 000000000..5f8c4ab94
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_gem.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_GEM_H
+#define PVRSGX_GEM_H
+
+#include <drm/drm_gem_shmem_helper.h>
+
+#include <linux/atomic.h>
+#include <linux/iosys-map.h>
+#include <linux/types.h>
+
+struct drm_device;
+struct drm_file;
+
+struct pvrsgx_bo {
+	struct drm_gem_shmem_object base;
+	u64 flags;
+	/** @code_end: A code object's end of the furthest upload (under the resv lock). */
+	u64 code_end;
+	/**
+	 * @code_abs: The code has absolute targets, checked for the object's start
+	 * at @code_place instructions from its USSE heap's base (resv lock).
+	 */
+	bool code_abs;
+	s64 code_place;
+	/**
+	 * @code_unmapping: A code object's mappings that are unmapped but that the
+	 * GPU may still fetch from: the jobs before the unmap or the BIF invalidate
+	 * after it are not done. UPLOAD_CODE treats them as live mappings.
+	 */
+	atomic_t code_unmapping;
+	/** @import_map: An import's CPU mapping, from its exporter, for the cache flushes. */
+	struct iosys_map import_map;
+};
+
+static inline struct pvrsgx_bo *to_pvrsgx_bo(struct drm_gem_object *obj)
+{
+	return container_of(to_drm_gem_shmem_obj(obj), struct pvrsgx_bo, base);
+}
+
+struct drm_gem_object *pvrsgx_gem_create_object(struct drm_device *drm, size_t size);
+int pvrsgx_ioctl_create_bo(struct drm_device *drm, void *data, struct drm_file *file);
+int pvrsgx_ioctl_get_bo_mmap_offset(struct drm_device *drm, void *data, struct drm_file *file);
+
+#endif /* PVRSGX_GEM_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.c
new file mode 100644
index 000000000..8b21315e9
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.c
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Kernel-owned objects at device-wide addresses: render contexts' and RT
+ * datasets' firmware structures, parameter buffers, free lists, region headers.
+ * The 3D parameters range (PB, free list, scratch, background block) is per VM
+ * instead: the DPM numbers PB pages from the heap base, so a second context's PB
+ * at a higher device-wide address hangs the GPU.
+ *
+ * Releasing one does not free it at once: a render already handed to the
+ * microkernel may still read it. Released objects are freed once every render
+ * submitted before the release has completed. A PB descriptor waits for the
+ * next restart instead: the microkernel remembers the last PB's address to
+ * skip reloading its free list, so a new PB at a freed one's address must not
+ * appear while it runs.
+ *
+ * Unmapping is not the end either: the BIF may hold a translation of the
+ * object's pages until it is invalidated, and the microkernel invalidates only
+ * for the cache control word of a command it takes, before it moves the read
+ * offset on. So a reaped object is unmapped, the invalidate is queued for the
+ * next command, and its pages and address are freed once the kernel CCB has
+ * consumed that command. A hardware unit's late write then faults instead of
+ * landing in a page the kernel has handed out again.
+ */
+
+#include <linux/mm.h>
+#include <linux/seq_buf.h>
+#include <linux/slab.h>
+
+#include "pvrsgx_ccb.h"
+#include "pvrsgx_kobj.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_mmu.h"
+#include "pvrsgx_vm.h"
+
+struct krange {
+	u32 start, end, pte;
+};
+
+/* Each range in the core's layout (pvrsgx_core.c). */
+static struct krange krange(const struct pvrsgx_device *sdev, enum pvrsgx_krange r)
+{
+	const struct pvrsgx_heap_layout *h = pvrsgx_heaps(sdev);
+
+	switch (r) {
+	case PVRSGX_KRANGE_KDATA:	/* past the firmware's own arenas (pvrsgx_fw.c) */
+		return (struct krange){ h->kernel_data.base + 0x10000, h->kernel_data.end,
+					SGX_MMU_PTE_CACHECONSISTENT | SGX_MMU_PTE_EDMPROTECT };
+	case PVRSGX_KRANGE_KCODE:
+		return (struct krange){ h->kernel_code.base + 0x10000, h->kernel_code.end,
+					SGX_MMU_PTE_CACHECONSISTENT };
+	case PVRSGX_KRANGE_SYNC:
+		return (struct krange){ h->sync.base, h->sync.end, SGX_MMU_PTE_CACHECONSISTENT };
+	case PVRSGX_KRANGE_3DPARAMS:
+		return (struct krange){ h->params_3d.base, h->params_3d.end, 0 };
+	case PVRSGX_KRANGE_TADATA:
+	default:
+		return (struct krange){ h->ta_data.base, h->ta_data.end, 0 };
+	}
+}
+
+int pvrsgx_kobj_init(struct pvrsgx_device *sdev)
+{
+	mutex_init(&sdev->kobj_lock);
+	INIT_LIST_HEAD(&sdev->graveyard);
+	INIT_LIST_HEAD(&sdev->kobj_unmapped);
+	for (int r = 0; r < PVRSGX_KRANGE_COUNT; r++) {
+		struct krange kr = krange(sdev, r);
+
+		drm_mm_init(&sdev->krange[r], kr.start, kr.end - kr.start);
+	}
+	return 0;
+}
+
+struct pvrsgx_kobj *pvrsgx_kobj_create(struct pvrsgx_device *sdev, enum pvrsgx_krange range,
+				       u32 size, u32 min_addr, struct pvrsgx_vm *vm)
+{
+	struct pvrsgx_kobj *o;
+	struct krange kr = krange(sdev, range);
+	u32 start = max(min_addr, kr.start);
+	/* A 3D-params object without a VM is the device's: the PB (pvrsgx_render.c). */
+	bool per_vm = range == PVRSGX_KRANGE_3DPARAMS && vm;
+	int err;
+
+	/* Clients' ioctls create these: charge them to the caller. */
+	o = kzalloc_obj(*o, GFP_KERNEL_ACCOUNT);
+	if (!o)
+		return ERR_PTR(-ENOMEM);
+	o->range = range;
+	o->mm = per_vm ? &vm->params : &sdev->krange[range];
+	mutex_lock(&sdev->kobj_lock);
+	err = drm_mm_insert_node_in_range(o->mm, &o->node, PAGE_ALIGN(size), PAGE_SIZE,
+					  0, start, kr.end, DRM_MM_INSERT_LOW);
+	mutex_unlock(&sdev->kobj_lock);
+	if (err)
+		goto err_free;
+
+	if (per_vm)
+		o->km.mmu = vm->mmu;
+	err = pvrsgx_kmem_alloc(sdev, &o->km, o->node.start, size, kr.pte,
+				__GFP_ACCOUNT);
+	if (err)
+		goto err_node;
+	memset(o->km.cpu, 0, o->km.size);
+
+	if (per_vm) {
+		o->vm = pvrsgx_vm_get(vm);
+	} else if (vm) {
+		err = pvrsgx_mmu_map_pages(vm->mmu, o->node.start, o->km.pages, o->km.npages, 0);
+		if (err)
+			goto err_kmem;
+		o->vm = pvrsgx_vm_get(vm);
+	}
+	mutex_lock(&sdev->ccb_lock);
+	sdev->fw.cache_control |= PVRSGX_CC_INVAL_BIF_PT | PVRSGX_CC_INVAL_BIF_PD;
+	mutex_unlock(&sdev->ccb_lock);
+	return o;
+
+err_kmem:
+	pvrsgx_kmem_free(sdev, &o->km);
+err_node:
+	mutex_lock(&sdev->kobj_lock);
+	drm_mm_remove_node(&o->node);
+	mutex_unlock(&sdev->kobj_lock);
+err_free:
+	kfree(o);
+	return ERR_PTR(err);
+}
+
+static void kobj_unmap(struct pvrsgx_kobj *o)
+{
+	struct pvrsgx_vm *vm = o->vm;
+
+	if (vm && o->km.mmu != vm->mmu)
+		pvrsgx_mmu_unmap(vm->mmu, kobj_va(o), o->km.size);
+	pvrsgx_mmu_unmap(o->km.mmu, o->km.dev_addr, o->km.size);
+}
+
+/* An unmapped object, after the invalidate: nothing can reach it now. */
+static void kobj_free(struct pvrsgx_device *sdev, struct pvrsgx_kobj *o)
+{
+	struct pvrsgx_vm *vm = o->vm;
+
+	pvrsgx_kmem_release(&o->km);
+	mutex_lock(&sdev->kobj_lock);
+	drm_mm_remove_node(&o->node);
+	mutex_unlock(&sdev->kobj_lock);
+	kfree(o);
+	/* Last: the node may belong to the VM's own allocator. */
+	pvrsgx_vm_put(vm);
+}
+
+void pvrsgx_kobj_release(struct pvrsgx_device *sdev, struct pvrsgx_kobj *o)
+{
+	if (IS_ERR_OR_NULL(o))
+		return;
+	mutex_lock(&sdev->kobj_lock);
+	o->grave_seq = sdev->renders_submitted;
+	o->grave_restarts = READ_ONCE(sdev->restarts);
+	list_add_tail(&o->grave, &sdev->graveyard);
+	mutex_unlock(&sdev->kobj_lock);
+}
+
+/*
+ * Unmap what was released before a render that has now finished, and free
+ * what was unmapped before a command the microkernel has since taken.
+ * renders_done U64_MAX (unbind, the GPU stopped) frees everything at once.
+ */
+void pvrsgx_kobj_reap(struct pvrsgx_device *sdev, u64 renders_done)
+{
+	struct pvrsgx_kobj *o, *tmp;
+	bool all = renders_done == U64_MAX;
+	u32 restarts = READ_ONCE(sdev->restarts);
+	u64 seq = 0, consumed;
+	LIST_HEAD(dead);
+
+	mutex_lock(&sdev->kobj_lock);
+	list_for_each_entry_safe(o, tmp, &sdev->graveyard, grave) {
+		/* A restart forgets everything the old run pointed at. */
+		if (all || o->grave_restarts != restarts ||
+		    (o->grave_seq < renders_done && !o->until_restart))
+			list_move_tail(&o->grave, &dead);
+	}
+	mutex_unlock(&sdev->kobj_lock);
+
+	if (!list_empty(&dead)) {
+		list_for_each_entry(o, &dead, grave)
+			kobj_unmap(o);
+		/* The next command carries the invalidate. */
+		mutex_lock(&sdev->ccb_lock);
+		sdev->fw.cache_control |= PVRSGX_CC_INVAL_BIF_PT | PVRSGX_CC_INVAL_BIF_PD;
+		seq = sdev->ccb_issued + 1;
+		mutex_unlock(&sdev->ccb_lock);
+	}
+
+	mutex_lock(&sdev->kobj_lock);
+	list_for_each_entry(o, &dead, grave)
+		o->flush_seq = seq;
+	list_splice_tail_init(&dead, &sdev->kobj_unmapped);
+	/*
+	 * After a restart the ring starts over empty, so everything sent before it
+	 * counts as consumed: the GPU was reset, and the first command after the
+	 * microkernel's start carries an invalidate of its own.
+	 */
+	consumed = all ? U64_MAX : pvrsgx_ccb_consumed(sdev);
+	list_for_each_entry_safe(o, tmp, &sdev->kobj_unmapped, grave) {
+		if (o->flush_seq <= consumed)
+			list_move_tail(&o->grave, &dead);
+	}
+	mutex_unlock(&sdev->kobj_lock);
+	list_for_each_entry_safe(o, tmp, &dead, grave)
+		kobj_free(sdev, o);
+}
+
+void pvrsgx_kobj_fini(struct pvrsgx_device *sdev)
+{
+	/*
+	 * Freeing can release more: a VM's last object frees the VM, which
+	 * drops its PB reference, and the PB's objects join the graveyard.
+	 */
+	for (;;) {
+		bool empty;
+
+		mutex_lock(&sdev->kobj_lock);
+		empty = list_empty(&sdev->graveyard) && list_empty(&sdev->kobj_unmapped);
+		mutex_unlock(&sdev->kobj_lock);
+		if (empty)
+			break;
+		pvrsgx_kobj_reap(sdev, U64_MAX);
+	}
+	for (int r = 0; r < PVRSGX_KRANGE_COUNT; r++)
+		drm_mm_takedown(&sdev->krange[r]);
+	mutex_destroy(&sdev->kobj_lock);
+}
+
+/* For status dumps: what waits to be freed. Never sleeps: false if busy. */
+bool pvrsgx_kobj_pending(struct pvrsgx_device *sdev, struct pvrsgx_kobj_counts *c)
+{
+	struct pvrsgx_kobj *o;
+
+	if (!mutex_trylock(&sdev->kobj_lock))
+		return false;
+	memset(c, 0, sizeof(*c));
+	list_for_each_entry(o, &sdev->graveyard, grave) {
+		c->graveyard++;
+		c->graveyard_pages += o->km.npages;
+	}
+	list_for_each_entry(o, &sdev->kobj_unmapped, grave) {
+		c->unmapped++;
+		c->unmapped_pages += o->km.npages;
+	}
+	mutex_unlock(&sdev->kobj_lock);
+	return true;
+}
+
+void pvrsgx_kobj_drop(struct pvrsgx_device *sdev, struct pvrsgx_kobj **o)
+{
+	pvrsgx_kobj_release(sdev, *o);
+	*o = NULL;
+}
+
+/* A kernel object into *dst, which stays NULL on failure. */
+int pvrsgx_kobj_new(struct pvrsgx_device *sdev, struct pvrsgx_kobj **dst,
+		    enum pvrsgx_krange range, u32 size, u32 min, struct pvrsgx_vm *vm)
+{
+	struct pvrsgx_kobj *o = pvrsgx_kobj_create(sdev, range, size, min, vm);
+
+	if (IS_ERR(o))
+		return PTR_ERR(o);
+	*dst = o;
+	return 0;
+}
+
+void pvrsgx_kobj_dump(struct seq_buf *sb, const char *name, struct pvrsgx_kobj *o, u32 off,
+		      u32 bytes)
+{
+	const u32 *w = (const u32 *)((const u8 *)kobj_cpu(o) + off);
+
+	seq_buf_printf(sb, "%s @ 0x%08x:", name, kobj_va(o) + off);
+	for (u32 i = 0; i < bytes / 4; i++)
+		seq_buf_printf(sb, "%s%08x", i % 8 ? " " : "\n  ", READ_ONCE(w[i]));
+	seq_buf_puts(sb, "\n");
+}
+
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.h
new file mode 100644
index 000000000..8af216e66
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_kobj.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_KOBJ_H
+#define PVRSGX_KOBJ_H
+
+#include <drm/drm_mm.h>
+
+#include <linux/list.h>
+#include <linux/types.h>
+
+#include "pvrsgx_device.h"
+
+struct pvrsgx_vm;
+
+/**
+ * struct pvrsgx_kobj - A kernel-owned object the firmware or the TA/3D reads
+ *
+ * Its address is unique across the device and the firmware's directory maps it,
+ * and a VM also maps the ones the TA and 3D use under its directory. The 3D
+ * parameters range is the exception: per VM, from the heap base, mapped only in
+ * that VM's directory. A client can never map anything in these ranges
+ * itself.
+ */
+struct pvrsgx_kobj {
+	/** @km: Its memory. */
+	struct pvrsgx_kmem km;
+	/** @node: Its place in @mm. */
+	struct drm_mm_node node;
+	/** @mm: The range allocator @node came from. */
+	struct drm_mm *mm;
+	/** @range: The range it was allocated in. */
+	enum pvrsgx_krange range;
+	/** @vm: Also mapped in this VM's directory, or NULL. */
+	struct pvrsgx_vm *vm;
+	/** @until_restart: Once released, kept until the next microkernel start. */
+	bool until_restart;
+	/** @grave: In pvrsgx_device.graveyard once released. */
+	struct list_head grave;
+	/** @grave_seq: pvrsgx_device.renders_submitted when it was released. */
+	u64 grave_seq;
+	/** @flush_seq: Once unmapped, the kernel CCB command whose cache control frees it. */
+	u64 flush_seq;
+	/** @grave_restarts: pvrsgx_device.restarts when it was released. */
+	u32 grave_restarts;
+};
+
+static inline u32 *kobj_cpu(struct pvrsgx_kobj *o)
+{
+	return o->km.cpu;
+}
+
+static inline u32 kobj_va(struct pvrsgx_kobj *o)
+{
+	return o->km.dev_addr;
+}
+
+int pvrsgx_kobj_init(struct pvrsgx_device *sdev);
+void pvrsgx_kobj_fini(struct pvrsgx_device *sdev);
+struct pvrsgx_kobj *pvrsgx_kobj_create(struct pvrsgx_device *sdev, enum pvrsgx_krange range,
+				       u32 size, u32 min_addr, struct pvrsgx_vm *vm);
+void pvrsgx_kobj_release(struct pvrsgx_device *sdev, struct pvrsgx_kobj *o);
+void pvrsgx_kobj_reap(struct pvrsgx_device *sdev, u64 renders_done);
+
+/* Objects and pages waiting in the graveyard, and unmapped ones waiting for the invalidate. */
+struct pvrsgx_kobj_counts {
+	u32 graveyard, unmapped;
+	u64 graveyard_pages, unmapped_pages;
+};
+
+bool pvrsgx_kobj_pending(struct pvrsgx_device *sdev, struct pvrsgx_kobj_counts *c);
+
+struct seq_buf;
+struct pvrsgx_vm;
+
+int pvrsgx_kobj_new(struct pvrsgx_device *sdev, struct pvrsgx_kobj **dst,
+		    enum pvrsgx_krange range, u32 size, u32 min, struct pvrsgx_vm *vm);
+void pvrsgx_kobj_drop(struct pvrsgx_device *sdev, struct pvrsgx_kobj **o);
+void pvrsgx_kobj_dump(struct seq_buf *sb, const char *name, struct pvrsgx_kobj *o, u32 off,
+		      u32 bytes);
+
+#endif /* PVRSGX_KOBJ_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c
new file mode 100644
index 000000000..e6aca1145
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Buffers imported from other devices. The GPU doesn't snoop the CPU caches
+ * and can only reach the low 4 GiB, so an import needs a reachability check
+ * when it is mapped, and cache flushes of the bytes each job lists
+ * (&drm_powervr_sgx_job.cache_ranges) around it.
+ */
+
+#include <drm/drm_cache.h>
+#include <drm/drm_file.h>
+#include <drm/drm_gem_shmem_helper.h>
+
+#include <linux/device.h>
+#include <linux/dma-buf.h>
+#include <linux/ktime.h>
+#include <linux/overflow.h>
+#include <linux/scatterlist.h>
+#include <linux/slab.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_drv.h"
+#include "pvrsgx_gem.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_prime.h"
+#include "pvrsgx_trace.h"
+
+/*
+ * The GPU's MMU takes 32-bit addresses. An import was mapped for our device,
+ * whose DMA mask is 32 bits, so the DMA API already keeps every address below
+ * 4 GiB (bouncing if it must, synced through the dma-buf's CPU access calls);
+ * check the addresses only, never the exporter's pages.
+ */
+bool pvrsgx_prime_reachable(struct sg_table *sgt)
+{
+	struct scatterlist *sg;
+	unsigned int i;
+
+	for_each_sgtable_dma_sg(sgt, sg, i) {
+		if (upper_32_bits(sg_dma_address(sg) + sg_dma_len(sg) - 1))
+			return false;
+	}
+	return true;
+}
+
+#define PVRSGX_MAX_CACHE_RANGES	64
+
+#define CACHE_RANGE_FLAGS	(DRM_POWERVR_SGX_CACHE_RANGE_FLAG_READ | \
+				 DRM_POWERVR_SGX_CACHE_RANGE_FLAG_WRITE)
+
+/* A range's flags, and its rows inside an object of obj_size bytes. */
+static int cache_range_check(const struct drm_powervr_sgx_cache_range *r, u64 obj_size)
+{
+	u32 rows = max(r->rows, 1u);
+	u64 span, end;
+
+	if (!r->flags || (r->flags & ~CACHE_RANGE_FLAGS) || !r->size)
+		return -EINVAL;
+	if (rows > 1 && r->stride < r->size)
+		return -EINVAL;
+	if (check_mul_overflow((u64)(rows - 1), (u64)r->stride, &span) ||
+	    check_add_overflow(span, r->size, &span) ||
+	    check_add_overflow(r->offset, span, &end) || end > obj_size)
+		return -EINVAL;
+	return 0;
+}
+
+/* The job's cache ranges, checked, each holding a reference on its object. */
+int pvrsgx_prime_ranges(struct drm_file *file, const struct drm_powervr_sgx_obj_array *arr,
+			struct pvrsgx_job *job)
+{
+	struct drm_powervr_sgx_cache_range *in;
+	int err = 0;
+
+	in = pvrsgx_obj_array_get(arr, sizeof(*in), PVRSGX_MAX_CACHE_RANGES);
+	if (IS_ERR_OR_NULL(in))
+		return PTR_ERR_OR_ZERO(in);
+	job->ranges = kvcalloc(arr->count, sizeof(*job->ranges), GFP_KERNEL);
+	if (!job->ranges) {
+		err = -ENOMEM;
+		goto out;
+	}
+	for (u32 i = 0; i < arr->count; i++) {
+		struct pvrsgx_cache_range *r = &job->ranges[i];
+
+		r->obj = drm_gem_object_lookup(file, in[i].handle);
+		if (!r->obj) {
+			err = -ENOENT;
+			break;
+		}
+		job->nranges++;
+		err = cache_range_check(&in[i], r->obj->size);
+		if (err)
+			break;
+		/*
+		 * Our own objects are write-combined: nothing to flush. A re-imported
+		 * dma-buf of ours comes back as such an object, so it isn't refused.
+		 */
+		r->import = drm_gem_is_imported(r->obj);
+		r->offset = in[i].offset;
+		r->size = in[i].size;
+		r->stride = in[i].stride;
+		r->rows = max(in[i].rows, 1u);
+		r->write = in[i].flags & DRM_POWERVR_SGX_CACHE_RANGE_FLAG_WRITE;
+	}
+out:
+	kvfree(in);
+	return err;
+}
+
+/*
+ * The GPU's accesses don't snoop the CPU caches, not even through
+ * cache-consistent PTEs. Our own objects are write-combined on the CPU; an
+ * import may be cached. Before a job every range it lists is written back and
+ * dropped (the GPU reads what the CPU wrote, no dirty line lands on its output
+ * later), and after it the ranges it wrote are dropped again (no stale line
+ * hides its output). An importer may not touch the attachment's pages, so the
+ * flushes go through the exporter's own CPU mapping, taken at import.
+ */
+static u64 range_flush(const struct pvrsgx_cache_range *r)
+{
+	const struct iosys_map *map = &to_pvrsgx_bo(r->obj)->import_map;
+
+	/* I/O memory isn't cached. */
+	if (!r->import || map->is_iomem)
+		return 0;
+	if (r->rows == 1 || r->stride == r->size) {
+		drm_clflush_virt_range(map->vaddr + r->offset, r->size * r->rows);
+	} else {
+		for (u32 y = 0; y < r->rows; y++)
+			drm_clflush_virt_range(map->vaddr + r->offset + (u64)y * r->stride,
+					       r->size);
+	}
+	return r->size * r->rows;
+}
+
+/* The whole of an import, at import. */
+static void import_flush(struct drm_gem_object *obj)
+{
+	const struct iosys_map *map = &to_pvrsgx_bo(obj)->import_map;
+
+	if (!map->is_iomem)
+		drm_clflush_virt_range(map->vaddr, obj->size);
+}
+
+void pvrsgx_prime_flush(struct pvrsgx_job *job, bool after)
+{
+	u64 t0 = trace_pvrsgx_flush_imports_enabled() ? ktime_get_ns() : 0, bytes = 0;
+
+	for (u32 i = 0; i < job->nranges; i++) {
+		if (!after || job->ranges[i].write)
+			bytes += range_flush(&job->ranges[i]);
+	}
+	if (job->nranges)
+		trace_pvrsgx_flush_imports(job->hw_fence ? job->hw_fence->seqno : 0,
+					   job->nranges, bytes,
+					   t0 ? ktime_get_ns() - t0 : 0, after);
+}
+
+struct drm_gem_object *pvrsgx_prime_import_sg_table(struct drm_device *drm,
+						    struct dma_buf_attachment *attach,
+						    struct sg_table *sgt)
+{
+	struct drm_gem_object *obj;
+	struct iosys_map map;
+	int err;
+
+	/* An import's flushes are CPU cache flushes of its vmap, which only x86 has here. */
+	if (!IS_ENABLED(CONFIG_X86))
+		return ERR_PTR(-EOPNOTSUPP);
+	/*
+	 * The exporter's mapping is taken once here and kept for the import's
+	 * life. A job's cache ranges are flushed through it in run_job and at
+	 * completion, inside the fence-signaling path, where a vmap can't be
+	 * taken: it may sleep and allocate. The importer never touches the
+	 * attachment's pages.
+	 */
+	err = dma_buf_vmap_unlocked(attach->dmabuf, &map);
+	if (err)
+		return ERR_PTR(err);
+	obj = drm_gem_shmem_prime_import_sg_table(drm, attach, sgt);
+	if (IS_ERR(obj)) {
+		dma_buf_vunmap_unlocked(attach->dmabuf, &map);
+		return obj;
+	}
+	to_pvrsgx_bo(obj)->import_map = map;
+	/*
+	 * Only the bytes a job lists are flushed around it: what the CPU still
+	 * holds of the rest (its zeroing, say) reaches memory now, or the GPU
+	 * could see the pages' earlier contents.
+	 */
+	import_flush(obj);
+	return obj;
+}
+
+void pvrsgx_prime_put(struct pvrsgx_job *job)
+{
+	for (u32 i = 0; i < job->nranges; i++)
+		drm_gem_object_put(job->ranges[i].obj);
+	kvfree(job->ranges);
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.h
new file mode 100644
index 000000000..dd0332f34
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_PRIME_H
+#define PVRSGX_PRIME_H
+
+#include <linux/types.h>
+
+struct dma_buf_attachment;
+struct drm_device;
+struct drm_file;
+struct drm_gem_object;
+struct drm_powervr_sgx_obj_array;
+struct pvrsgx_job;
+struct sg_table;
+
+bool pvrsgx_prime_reachable(struct sg_table *sgt);
+struct drm_gem_object *pvrsgx_prime_import_sg_table(struct drm_device *drm,
+						    struct dma_buf_attachment *attach,
+						    struct sg_table *sgt);
+int pvrsgx_prime_ranges(struct drm_file *file, const struct drm_powervr_sgx_obj_array *arr,
+			struct pvrsgx_job *job);
+void pvrsgx_prime_flush(struct pvrsgx_job *job, bool after);
+void pvrsgx_prime_put(struct pvrsgx_job *job);
+
+#endif /* PVRSGX_PRIME_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_vm.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_vm.c
new file mode 100644
index 000000000..b12d39062
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_vm.c
@@ -0,0 +1,799 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * VM contexts on drm_gpuvm. The client picks every address inside the heaps
+ * DEV_QUERY reports. VM_BIND maps and unmaps through drm_gpuvm's split and
+ * merge steps; an unmap takes the page tables down at once and lets go of the
+ * objects once every job submitted before it is done, from a fence callback.
+ */
+
+#include <drm/drm_drv.h>
+#include <drm/drm_exec.h>
+#include <drm/drm_gem.h>
+#include <drm/drm_print.h>
+#include <drm/drm_syncobj.h>
+
+#include <linux/device.h>
+#include <linux/dma-resv.h>
+#include <linux/module.h>
+#include <linux/overflow.h>
+#include <linux/pm_runtime.h>
+#include <linux/scatterlist.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_ccb.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_drv.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_gem.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_mmu.h"
+#include "pvrsgx_render.h"
+#include "pvrsgx_validate.h"
+#include "pvrsgx_prime.h"
+#include "pvrsgx_vm.h"
+
+/* The client heaps, the same in every VM, from the core's layout. */
+static void client_heaps(const struct pvrsgx_device *sdev,
+			 struct drm_powervr_sgx_heap heaps[DRM_POWERVR_SGX_HEAP_COUNT])
+{
+	const struct pvrsgx_heap_layout *h = pvrsgx_heaps(sdev);
+	const struct {
+		const struct pvrsgx_va_range *r;
+		u32 usse_base_index;
+	} from[DRM_POWERVR_SGX_HEAP_COUNT] = {
+		[DRM_POWERVR_SGX_HEAP_GENERAL] = { &h->general },
+		[DRM_POWERVR_SGX_HEAP_PDS_PIXEL] = { &h->pds_pixel },
+		[DRM_POWERVR_SGX_HEAP_PDS_VERTEX] = { &h->pds_vertex },
+		[DRM_POWERVR_SGX_HEAP_USSE_PIXEL] = { &h->usse_pixel,
+						      PVRSGX_USSE_PIXEL_BASE_INDEX },
+		[DRM_POWERVR_SGX_HEAP_USSE_VERTEX] = { &h->usse_vertex,
+						       PVRSGX_USSE_VERTEX_BASE_INDEX },
+		[DRM_POWERVR_SGX_HEAP_TEXTURE] = { &h->texture },
+	};
+
+	for (int i = 0; i < DRM_POWERVR_SGX_HEAP_COUNT; i++) {
+		bool usse = i == DRM_POWERVR_SGX_HEAP_USSE_PIXEL ||
+			    i == DRM_POWERVR_SGX_HEAP_USSE_VERTEX;
+
+		heaps[i] = (struct drm_powervr_sgx_heap){
+			.base = from[i].r->base,
+			.size = from[i].r->end - from[i].r->base,
+			.page_size_log2 = SGX_MMU_PAGE_SHIFT,
+			.usse_base_index = from[i].usse_base_index,
+			.usse_window_log2 = usse ? PVRSGX_USSE_WINDOW_LOG2 : 0,
+		};
+	}
+}
+
+/* The heap holding all of [addr, addr + size), or -1. */
+int pvrsgx_heap_find(const struct pvrsgx_device *sdev, u64 addr, u64 size)
+{
+	struct drm_powervr_sgx_heap heaps[DRM_POWERVR_SGX_HEAP_COUNT];
+	u64 end;
+
+	if (check_add_overflow(addr, size, &end))
+		return -1;
+	client_heaps(sdev, heaps);
+	for (int i = 0; i < ARRAY_SIZE(heaps); i++) {
+		if (addr >= heaps[i].base && end <= heaps[i].base + heaps[i].size)
+			return i;
+	}
+	return -1;
+}
+
+int pvrsgx_query_heap_info(struct pvrsgx_device *sdev, void __user *ptr, u32 *size)
+{
+	struct drm_powervr_sgx_heap heaps[DRM_POWERVR_SGX_HEAP_COUNT];
+	struct drm_powervr_sgx_dev_query_heap_info info = {};
+	int err;
+
+	if (!ptr) {
+		*size = sizeof(info);
+		return 0;
+	}
+	err = copy_struct_from_user(&info, sizeof(info), ptr, *size);
+	if (err)
+		return err;
+
+	client_heaps(sdev, heaps);
+	if (info.heaps.array) {
+		err = pvrsgx_obj_array_put(&info.heaps, heaps, sizeof(heaps[0]), ARRAY_SIZE(heaps));
+		if (err)
+			return err;
+	}
+	info.heaps.stride = sizeof(heaps[0]);
+	info.heaps.count = ARRAY_SIZE(heaps);
+
+	*size = min_t(u32, *size, sizeof(info));
+	return copy_to_user(ptr, &info, *size) ? -EFAULT : 0;
+}
+
+struct pvrsgx_vm *pvrsgx_vm_get(struct pvrsgx_vm *vm)
+{
+	kref_get(&vm->ref);
+	return vm;
+}
+
+struct dma_resv *pvrsgx_vm_resv(struct pvrsgx_vm *vm)
+{
+	return drm_gpuvm_resv(&vm->gpuvm);
+}
+
+static const struct drm_gpuvm_ops pvrsgx_gpuvm_ops;
+
+/* A mapping. Once unmapped, it keeps its object until the jobs before the unmap are done. */
+struct pvrsgx_va {
+	struct drm_gpuva base;
+	struct drm_gem_object *obj;
+	struct list_head free_link;
+};
+
+static struct pvrsgx_va *to_pvrsgx_va(struct drm_gpuva *va)
+{
+	return container_of(va, struct pvrsgx_va, base);
+}
+
+/*
+ * What an unmap or a teardown lets go of once every job on the VM so far is
+ * done. A teardown also takes the mappings down then.
+ */
+struct pvrsgx_vm_free {
+	struct dma_fence_cb cb;
+	struct work_struct work;
+	struct pvrsgx_inval_wait inval;
+	struct drm_device *drm;
+	struct dma_fence *fence;
+	struct list_head vas;
+	struct pvrsgx_vm *vm;
+	bool teardown;
+};
+
+/* The BIF may hold walks of what changed: the next command drops them. */
+static void vm_invalidate(struct pvrsgx_device *sdev, bool data)
+{
+	mutex_lock(&sdev->ccb_lock);
+	sdev->fw.cache_control |= PVRSGX_CC_INVAL_BIF_PT | PVRSGX_CC_INVAL_BIF_PD |
+				  (data ? PVRSGX_CC_INVAL_DATA : 0);
+	mutex_unlock(&sdev->ccb_lock);
+}
+
+/* A code object counts its unmapped mappings until vas_free() (pvrsgx_bo.code_unmapping). */
+static bool va_obj_is_code(struct pvrsgx_va *v)
+{
+	return to_pvrsgx_bo(v->obj)->flags & DRM_POWERVR_SGX_BO_CODE;
+}
+
+/* Unlink a removed mapping from its object, keeping a reference on freed. */
+static void va_unlink_locked(struct drm_gpuva *va, struct list_head *freed)
+{
+	struct pvrsgx_va *v = to_pvrsgx_va(va);
+
+	v->obj = va->gem.obj;
+	drm_gem_object_get(v->obj);
+	if (va_obj_is_code(v))
+		atomic_inc(&to_pvrsgx_bo(v->obj)->code_unmapping);
+	drm_gpuva_unlink(va);
+	list_add_tail(&v->free_link, freed);
+}
+
+static void vas_free(struct list_head *list)
+{
+	struct pvrsgx_va *v, *tmp;
+
+	list_for_each_entry_safe(v, tmp, list, free_link) {
+		list_del(&v->free_link);
+		if (va_obj_is_code(v))
+			atomic_dec(&to_pvrsgx_bo(v->obj)->code_unmapping);
+		drm_gem_object_put(v->obj);
+		kfree(v);
+	}
+}
+
+/* Every mapping down, onto freed. Nothing may submit to the VM any more. */
+static void vm_unmap_all(struct pvrsgx_vm *vm, struct list_head *freed)
+{
+	struct drm_gpuva *va, *next;
+
+	drm_gpuvm_for_each_va_safe(va, next, &vm->gpuvm) {
+		struct drm_gem_object *obj = va->gem.obj;
+
+		pvrsgx_mmu_unmap(vm->mmu, va->va.addr, va->va.range);
+		drm_gpuva_remove(va);
+		dma_resv_lock(obj->resv, NULL);
+		va_unlink_locked(va, freed);
+		dma_resv_unlock(obj->resv);
+	}
+}
+
+/* The invalidate has run: nothing can reach the unmapped pages any more. */
+static void vm_free_done(struct pvrsgx_inval_wait *w)
+{
+	struct pvrsgx_vm_free *f = container_of(w, struct pvrsgx_vm_free, inval);
+
+	vas_free(&f->vas);
+	pvrsgx_vm_put(f->vm);
+	dma_fence_put(f->fence);
+	drm_dev_put(f->drm);
+	kfree(f);
+}
+
+static void vm_free_work(struct work_struct *work)
+{
+	struct pvrsgx_vm_free *f = container_of(work, struct pvrsgx_vm_free, work);
+	struct pvrsgx_device *sdev = to_pvrsgx_device(f->drm);
+
+	f->inval.done = vm_free_done;
+	if (f->teardown) {
+		mutex_lock(&f->vm->lock);
+		vm_unmap_all(f->vm, &f->vas);
+		mutex_unlock(&f->vm->lock);
+	}
+	/*
+	 * The BIF caches walks only of a directory a job ran on, and a restart or
+	 * power-off drops them: then there is nothing to wait for.
+	 */
+	if (!READ_ONCE(sdev->powered) ||
+	    READ_ONCE(f->vm->ran_gen) != READ_ONCE(sdev->fw_gen) + 1) {
+		vm_free_done(&f->inval);
+		return;
+	}
+	pvrsgx_ccb_after_invalidate(sdev, &f->inval, PVRSGX_CC_INVAL_BIF_PT |
+				    PVRSGX_CC_INVAL_BIF_PD | PVRSGX_CC_INVAL_DATA);
+}
+
+static void vm_free_cb(struct dma_fence *fence, struct dma_fence_cb *cb)
+{
+	struct pvrsgx_vm_free *f = container_of(cb, struct pvrsgx_vm_free, cb);
+
+	queue_work(system_dfl_wq, &f->work);
+}
+
+/*
+ * Run f once every job submitted to vm so far is done, without waiting for
+ * them. With none left, at once if now is set (the caller holds no VM lock).
+ */
+static void vm_free_after_jobs(struct pvrsgx_vm *vm, struct pvrsgx_vm_free *f, bool now)
+{
+	struct dma_fence *fence = NULL;
+
+	f->drm = &vm->sdev->base;
+	drm_dev_get(f->drm);
+	f->vm = pvrsgx_vm_get(vm);
+	INIT_WORK(&f->work, vm_free_work);
+	if (dma_resv_get_singleton(pvrsgx_vm_resv(vm), DMA_RESV_USAGE_BOOKKEEP, &fence)) {
+		/* No memory for a fence array: wait here instead. */
+		dma_resv_wait_timeout(pvrsgx_vm_resv(vm), DMA_RESV_USAGE_BOOKKEEP, false,
+				      MAX_SCHEDULE_TIMEOUT);
+		fence = NULL;
+	}
+	f->fence = fence;
+	if (fence && !dma_fence_add_callback(fence, &f->cb, vm_free_cb))
+		return;
+	if (now)
+		vm_free_work(&f->work);
+	else
+		queue_work(system_dfl_wq, &f->work);
+}
+
+static struct pvrsgx_vm_free *vm_free_alloc(void)
+{
+	struct pvrsgx_vm_free *f = kzalloc_obj(*f);
+
+	if (f)
+		INIT_LIST_HEAD(&f->vas);
+	return f;
+}
+
+/*
+ * On a single-context core the microkernel puts list 0 back on the firmware's
+ * directory when it raises its interrupt, which can come after a job's
+ * completion shows. Before a VM's directory is freed, list 0 must not hold it.
+ * A suspended core holds nothing: the next start's reset sets list 0.
+ */
+static void vm_pd_unload(struct pvrsgx_device *sdev, struct pvrsgx_mmu *mmu)
+{
+	struct device *dev = sdev->base.dev;
+	u32 pd = pvrsgx_mmu_pd_addr(mmu);
+
+	if (!pvrsgx_core_single_context(sdev->family))
+		return;
+	if (IS_ENABLED(CONFIG_PM) && pm_runtime_get_if_active(dev) <= 0)
+		return;
+	if (READ_ONCE(sdev->powered) && sgx_rd(sdev, SGX_CR_BIF_DIR_LIST_BASE0) == pd) {
+		drm_dbg_driver(&sdev->base, "list 0 still on a released VM's directory\n");
+		sgx_wr(sdev, SGX_CR_BIF_DIR_LIST_BASE0, pvrsgx_mmu_pd_addr(sdev->kernel_mmu));
+	}
+	if (IS_ENABLED(CONFIG_PM))
+		pm_runtime_put(dev);
+}
+
+static void vm_release(struct kref *ref)
+{
+	struct pvrsgx_vm *vm = container_of(ref, struct pvrsgx_vm, ref);
+	LIST_HEAD(freed);
+
+	/* No job holds this VM any more, so nothing on the GPU uses it. */
+	vm_unmap_all(vm, &freed);
+	vas_free(&freed);
+	if (vm->pb)
+		pvrsgx_pb_unmap(vm);
+	drm_mm_takedown(&vm->params);
+	if (vm->pd_slot)
+		pvrsgx_fw_pd_slot_put(vm->sdev, vm->pd_slot);
+	vm_pd_unload(vm->sdev, vm->mmu);
+	pvrsgx_mmu_destroy(vm->mmu);
+	/* The directory and its pages may come back as another VM's. */
+	mutex_lock(&vm->sdev->ccb_lock);
+	vm->sdev->fw.cache_control |= PVRSGX_CC_INVAL_BIF_PT | PVRSGX_CC_INVAL_BIF_PD |
+				      PVRSGX_CC_INVAL_DATA;
+	mutex_unlock(&vm->sdev->ccb_lock);
+	mutex_destroy(&vm->lock);
+	drm_gpuvm_put(&vm->gpuvm);
+}
+
+void pvrsgx_vm_put(struct pvrsgx_vm *vm)
+{
+	if (vm)
+		kref_put(&vm->ref, vm_release);
+}
+
+void pvrsgx_vm_user_get(struct pvrsgx_vm *vm)
+{
+	atomic_inc(&vm->users);
+}
+
+/*
+ * The handle or a context is gone. With neither left, nothing can submit to
+ * the VM again: once its jobs are done, unmap and release every object. The
+ * VM itself can live on much longer, held by its contexts' kernel objects
+ * until a later render (pvrsgx_kobj.c), but the client's memory goes then.
+ */
+void pvrsgx_vm_user_put(struct pvrsgx_vm *vm)
+{
+	struct pvrsgx_vm_free *f;
+	LIST_HEAD(freed);
+
+	if (!atomic_dec_and_test(&vm->users))
+		return;
+
+	f = vm_free_alloc();
+	if (f) {
+		f->teardown = true;
+		vm_free_after_jobs(vm, f, false);
+		return;
+	}
+	/*
+	 * No memory to defer it: wait for the jobs here. The pages then go before
+	 * the invalidate has run; nothing of this VM can run again, and the next
+	 * scene's directory switch invalidates before any translation.
+	 */
+	dma_resv_wait_timeout(pvrsgx_vm_resv(vm), DMA_RESV_USAGE_BOOKKEEP, false,
+			      MAX_SCHEDULE_TIMEOUT);
+	mutex_lock(&vm->lock);
+	vm_unmap_all(vm, &freed);
+	mutex_unlock(&vm->lock);
+	vm_invalidate(vm->sdev, true);
+	vas_free(&freed);
+}
+
+struct pvrsgx_vm *pvrsgx_vm_create(struct pvrsgx_device *sdev)
+{
+	struct drm_gem_object *r_obj;
+	struct pvrsgx_vm *vm;
+
+	vm = kzalloc_obj(*vm);
+	if (!vm)
+		return ERR_PTR(-ENOMEM);
+	vm->mmu = pvrsgx_mmu_create(sdev);
+	if (IS_ERR(vm->mmu)) {
+		struct pvrsgx_mmu *mmu = vm->mmu;
+
+		kfree(vm);
+		return ERR_CAST(mmu);
+	}
+	r_obj = drm_gpuvm_resv_object_alloc(&sdev->base);
+	if (!r_obj) {
+		pvrsgx_mmu_destroy(vm->mmu);
+		kfree(vm);
+		return ERR_PTR(-ENOMEM);
+	}
+	if (pvrsgx_core_single_context(sdev->family)) {
+		pvrsgx_share_fw_heaps(sdev, vm->mmu);
+		vm->pd_slot = pvrsgx_fw_pd_slot_get(sdev, pvrsgx_mmu_pd_addr(vm->mmu));
+		if (vm->pd_slot < 0) {
+			int err = vm->pd_slot;
+
+			drm_gem_object_put(r_obj);
+			pvrsgx_mmu_destroy(vm->mmu);
+			kfree(vm);
+			return ERR_PTR(err);
+		}
+	}
+	drm_gpuvm_init(&vm->gpuvm, "powervr_sgx", 0, &sdev->base, r_obj,
+		       0, BIT_ULL(sdev->family->va_bits), 0, 0, &pvrsgx_gpuvm_ops);
+	drm_gem_object_put(r_obj);
+	vm->sdev = sdev;
+	drm_mm_init(&vm->params, pvrsgx_heaps(sdev)->params_3d.base,
+		    pvrsgx_heaps(sdev)->params_3d.end - pvrsgx_heaps(sdev)->params_3d.base);
+	mutex_init(&vm->lock);
+	kref_init(&vm->ref);
+	atomic_set(&vm->users, 1);
+	return vm;
+}
+
+/*
+ * What a HW context's directory word holds: the directory's address, or on a
+ * single-context core its slot in the microkernel's PD table.
+ */
+u32 pvrsgx_vm_pd_word(const struct pvrsgx_vm *vm)
+{
+	return vm->pd_slot ?: pvrsgx_mmu_pd_addr(vm->mmu);
+}
+
+struct pvrsgx_vm *pvrsgx_vm_lookup(struct pvrsgx_file *sfile, u32 handle)
+{
+	struct pvrsgx_vm *vm;
+
+	xa_lock(&sfile->vms);
+	vm = xa_load(&sfile->vms, handle);
+	if (vm)
+		pvrsgx_vm_get(vm);
+	xa_unlock(&sfile->vms);
+	return vm;
+}
+
+void pvrsgx_vm_destroy_all(struct pvrsgx_file *sfile)
+{
+	struct pvrsgx_vm *vm;
+	unsigned long handle;
+
+	xa_for_each(&sfile->vms, handle, vm) {
+		xa_erase(&sfile->vms, handle);
+		pvrsgx_vm_user_put(vm);
+		pvrsgx_vm_put(vm);
+	}
+}
+
+int pvrsgx_ioctl_create_vm_context(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_create_vm_context_args *args = data;
+	struct pvrsgx_file *sfile = to_pvrsgx_file(file);
+	struct pvrsgx_vm *vm;
+	int err;
+
+	if (args->_padding_4)
+		return -EINVAL;
+
+	vm = pvrsgx_vm_create(to_pvrsgx_device(drm));
+	if (IS_ERR(vm))
+		return PTR_ERR(vm);
+	err = xa_alloc(&sfile->vms, &args->handle, vm, xa_limit_32b, GFP_KERNEL);
+	if (err)
+		pvrsgx_vm_put(vm);
+	return err;
+}
+
+int pvrsgx_ioctl_destroy_vm_context(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_destroy_vm_context_args *args = data;
+	struct pvrsgx_vm *vm;
+
+	if (args->_padding_4)
+		return -EINVAL;
+
+	vm = xa_erase(&to_pvrsgx_file(file)->vms, args->handle);
+	if (!vm)
+		return -ENOENT;
+	/* Contexts and jobs that use it keep it until they are done. */
+	pvrsgx_vm_user_put(vm);
+	pvrsgx_vm_put(vm);
+	return 0;
+}
+
+#define PVRSGX_MAX_BIND_OPS	256
+
+/* One op of a VM_BIND and what it needs, all gathered before anything changes. */
+struct bind_op {
+	struct drm_powervr_sgx_vm_bind_op in;
+	/* MAP: the object, its pages, its link to the VM, the new mapping. */
+	struct drm_gem_object *obj;
+	struct sg_table *sgt;
+	struct drm_gpuvm_bo *vm_bo;
+	/* MAP: [0]. UNMAP: the pieces a split may leave at either end. */
+	struct pvrsgx_va *va[2];
+};
+
+struct bind_ctx {
+	struct pvrsgx_vm *vm;
+	struct bind_op *op;
+	int next_va;
+	struct list_head *freed;
+};
+
+static struct pvrsgx_va *bind_take_va(struct bind_ctx *b)
+{
+	struct pvrsgx_va *v = b->op->va[b->next_va];
+
+	b->op->va[b->next_va++] = NULL;
+	return v;
+}
+
+/* The steps below can't fail: bind_prepare() checked and allocated everything. */
+static int bind_step_map(struct drm_gpuva_op *op, void *priv)
+{
+	struct bind_ctx *b = priv;
+	struct bind_op *o = b->op;
+	struct pvrsgx_va *v = bind_take_va(b);
+
+	WARN_ON(pvrsgx_mmu_map_sgt(b->vm->mmu, op->map.va.addr, o->sgt, op->map.gem.offset,
+				   op->map.va.range,
+				   pvrsgx_validate_pte_flags(to_pvrsgx_bo(o->obj))));
+	drm_gpuva_map(&b->vm->gpuvm, &v->base, &op->map);
+	drm_gpuva_link(&v->base, o->vm_bo);
+	return 0;
+}
+
+static int bind_step_remap(struct drm_gpuva_op *op, void *priv)
+{
+	struct bind_ctx *b = priv;
+	struct drm_gpuva *va = op->remap.unmap->va;
+	struct pvrsgx_va *prev = NULL, *next = NULL;
+	u64 addr, range;
+
+	drm_gpuva_op_remap_to_unmap_range(&op->remap, &addr, &range);
+	pvrsgx_mmu_unmap(b->vm->mmu, addr, range);
+	if (op->remap.prev)
+		prev = bind_take_va(b);
+	if (op->remap.next)
+		next = bind_take_va(b);
+	drm_gpuva_remap(prev ? &prev->base : NULL, next ? &next->base : NULL, &op->remap);
+	if (prev)
+		drm_gpuva_link(&prev->base, va->vm_bo);
+	if (next)
+		drm_gpuva_link(&next->base, va->vm_bo);
+	va_unlink_locked(va, b->freed);
+	return 0;
+}
+
+static int bind_step_unmap(struct drm_gpuva_op *op, void *priv)
+{
+	struct bind_ctx *b = priv;
+	struct drm_gpuva *va = op->unmap.va;
+
+	pvrsgx_mmu_unmap(b->vm->mmu, va->va.addr, va->va.range);
+	drm_gpuva_unmap(&op->unmap);
+	va_unlink_locked(va, b->freed);
+	return 0;
+}
+
+static void pvrsgx_gpuvm_free(struct drm_gpuvm *gpuvm)
+{
+	kfree(container_of(gpuvm, struct pvrsgx_vm, gpuvm));
+}
+
+static const struct drm_gpuvm_ops pvrsgx_gpuvm_ops = {
+	.vm_free = pvrsgx_gpuvm_free,
+	.sm_step_map = bind_step_map,
+	.sm_step_remap = bind_step_remap,
+	.sm_step_unmap = bind_step_unmap,
+};
+
+static bool ranges_overlap(u64 a, u64 as, u64 b, u64 bs)
+{
+	return a < b + bs && b < a + as;
+}
+
+/* The op's own arguments, and its object for a map. */
+static int bind_op_check(struct drm_file *file, struct pvrsgx_vm *vm, struct bind_op *o)
+{
+	const struct drm_powervr_sgx_vm_bind_op *in = &o->in;
+	u64 end;
+	int heap, err;
+
+	if (in->flags || in->_padding_24 || !in->size || !PAGE_ALIGNED(in->size) ||
+	    !PAGE_ALIGNED(in->device_addr) || check_add_overflow(in->device_addr, in->size, &end) ||
+	    end > BIT_ULL(vm->sdev->family->va_bits))
+		return -EINVAL;
+	if (in->op == DRM_POWERVR_SGX_VM_BIND_OP_UNMAP)
+		return in->handle || in->bo_offset ? -EINVAL : 0;
+	if (in->op != DRM_POWERVR_SGX_VM_BIND_OP_MAP || !PAGE_ALIGNED(in->bo_offset))
+		return -EINVAL;
+	heap = pvrsgx_heap_find(vm->sdev, in->device_addr, in->size);
+	if (heap < 0)
+		return -EINVAL;
+
+	o->obj = drm_gem_object_lookup(file, in->handle);
+	if (!o->obj)
+		return -ENOENT;
+	if (check_add_overflow(in->bo_offset, in->size, &end) || end > o->obj->size)
+		return -EINVAL;
+	err = pvrsgx_validate_vm_map(vm->sdev, vm, heap, to_pvrsgx_bo(o->obj), in->bo_offset,
+				     in->size);
+	if (err)
+		return err;
+	o->sgt = drm_gem_shmem_get_pages_sgt(to_drm_gem_shmem_obj(o->obj));
+	if (IS_ERR(o->sgt)) {
+		err = PTR_ERR(o->sgt);
+		o->sgt = NULL;
+		return err;
+	}
+	if (drm_gem_is_imported(o->obj) && !pvrsgx_prime_reachable(o->sgt))
+		return -EFAULT;
+	return 0;
+}
+
+/*
+ * With vm->lock and every object involved locked: check each op against the
+ * VM as it is (the ops don't overlap, so none depends on another) and allocate
+ * what applying it takes.
+ */
+static int bind_prepare(struct pvrsgx_vm *vm, struct bind_op *ops, u32 n)
+{
+	for (u32 i = 0; i < n; i++) {
+		struct bind_op *o = &ops[i];
+		u64 addr = o->in.device_addr, size = o->in.size;
+		int nva = o->in.op == DRM_POWERVR_SGX_VM_BIND_OP_MAP ? 1 : 2;
+		int err;
+
+		if (o->in.op == DRM_POWERVR_SGX_VM_BIND_OP_MAP) {
+			if (drm_gpuva_find_first(&vm->gpuvm, addr, size))
+				return -EEXIST;
+			err = pvrsgx_validate_vm_link(to_pvrsgx_bo(o->obj), addr, o->in.bo_offset);
+			if (err)
+				return err;
+			err = pvrsgx_mmu_reserve(vm->mmu, addr, size);
+			if (err)
+				return err;
+			if (!o->vm_bo) {
+				o->vm_bo = drm_gpuvm_bo_obtain_locked(&vm->gpuvm, o->obj);
+				if (IS_ERR(o->vm_bo)) {
+					err = PTR_ERR(o->vm_bo);
+					o->vm_bo = NULL;
+					return err;
+				}
+			}
+		}
+		for (int k = 0; k < nva; k++) {
+			if (!o->va[k])
+				o->va[k] = kzalloc_obj(*o->va[k]);
+			if (!o->va[k])
+				return -ENOMEM;
+		}
+	}
+	return 0;
+}
+
+static void bind_apply(struct pvrsgx_vm *vm, struct bind_op *ops, u32 n, struct list_head *freed)
+{
+	struct bind_ctx b = { .vm = vm, .freed = freed };
+
+	for (u32 i = 0; i < n; i++) {
+		struct bind_op *o = &ops[i];
+
+		b.op = o;
+		b.next_va = 0;
+		if (o->in.op == DRM_POWERVR_SGX_VM_BIND_OP_MAP) {
+			struct drm_gpuvm_map_req req = {
+				.map.va.addr = o->in.device_addr,
+				.map.va.range = o->in.size,
+				.map.gem.obj = o->obj,
+				.map.gem.offset = o->in.bo_offset,
+			};
+
+			WARN_ON(drm_gpuvm_sm_map(&vm->gpuvm, &b, &req));
+		} else {
+			WARN_ON(drm_gpuvm_sm_unmap(&vm->gpuvm, &b, o->in.device_addr, o->in.size));
+		}
+	}
+}
+
+/* Wait, interruptibly, for every wait op; find every signal op's syncobj. */
+int pvrsgx_ioctl_vm_bind(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_vm_bind_args *args = data;
+	struct pvrsgx_vm_free *f = NULL;
+	struct bind_op *ops = NULL;
+	struct drm_powervr_sgx_vm_bind_op *in;
+	struct pvrsgx_vm *vm;
+	struct drm_exec exec;
+	u32 n = args->ops.count;
+	int err;
+
+	/* A synchronous bind takes no fences (drm-vm-bind-async.rst). */
+	if (args->flags != DRM_POWERVR_SGX_VM_BIND_FLAG_SYNC || !n || args->sync_ops.count)
+		return -EINVAL;
+
+	vm = pvrsgx_vm_lookup(to_pvrsgx_file(file), args->vm_context_handle);
+	if (!vm)
+		return -ENOENT;
+	in = pvrsgx_obj_array_get(&args->ops, sizeof(*in), PVRSGX_MAX_BIND_OPS);
+	if (IS_ERR(in)) {
+		err = PTR_ERR(in);
+		goto out_vm;
+	}
+	ops = kvcalloc(n, sizeof(*ops), GFP_KERNEL);
+	f = vm_free_alloc();
+	if (!ops || !f) {
+		err = -ENOMEM;
+		goto out;
+	}
+	for (u32 i = 0; i < n; i++) {
+		ops[i].in = in[i];
+		err = bind_op_check(file, vm, &ops[i]);
+		if (err)
+			goto out;
+		for (u32 k = 0; k < i; k++) {
+			if (ranges_overlap(in[i].device_addr, in[i].size, in[k].device_addr,
+					   in[k].size)) {
+				err = -EINVAL;
+				goto out;
+			}
+		}
+	}
+
+	mutex_lock(&vm->lock);
+	drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT | DRM_EXEC_IGNORE_DUPLICATES, 0);
+	drm_exec_until_all_locked(&exec) {
+		err = 0;
+		for (u32 i = 0; i < n && !err; i++) {
+			struct drm_gpuva *va;
+
+			if (ops[i].obj) {
+				err = drm_exec_lock_obj(&exec, ops[i].obj);
+				drm_exec_retry_on_contention(&exec);
+				continue;
+			}
+			drm_gpuvm_for_each_va_range(va, &vm->gpuvm, in[i].device_addr,
+						    in[i].device_addr + in[i].size) {
+				err = drm_exec_lock_obj(&exec, va->gem.obj);
+				drm_exec_retry_on_contention(&exec);
+				if (err)
+					break;
+			}
+		}
+	}
+	err = err ?: bind_prepare(vm, ops, n);
+	if (!err)
+		bind_apply(vm, ops, n, &f->vas);
+	drm_exec_fini(&exec);
+	mutex_unlock(&vm->lock);
+	if (err)
+		goto out;
+
+	vm_invalidate(vm->sdev, false);
+	/*
+	 * The objects unmapped stay until every job submitted before now is done
+	 * and, if the VM ran one since the GPU started, the BIF invalidate has run.
+	 */
+	if (!list_empty(&f->vas)) {
+		vm_free_after_jobs(vm, f, true);
+		f = NULL;
+	}
+
+out:
+	if (ops) {
+		for (u32 i = 0; i < n; i++) {
+			kfree(ops[i].va[0]);
+			kfree(ops[i].va[1]);
+			if (ops[i].vm_bo) {
+				dma_resv_lock(ops[i].obj->resv, NULL);
+				drm_gpuvm_bo_put(ops[i].vm_bo);
+				dma_resv_unlock(ops[i].obj->resv);
+			}
+			if (ops[i].obj)
+				drm_gem_object_put(ops[i].obj);
+		}
+		kvfree(ops);
+	}
+	kfree(f);
+	kvfree(in);
+out_vm:
+	pvrsgx_vm_put(vm);
+	return err;
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_vm.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_vm.h
new file mode 100644
index 000000000..9a9a95262
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_vm.h
@@ -0,0 +1,69 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_VM_H
+#define PVRSGX_VM_H
+
+#include <drm/drm_gpuvm.h>
+#include <drm/drm_mm.h>
+
+#include <linux/kref.h>
+#include <linux/mutex.h>
+#include <linux/types.h>
+
+struct drm_device;
+struct drm_file;
+struct pvrsgx_device;
+struct pvrsgx_file;
+struct pvrsgx_pb;
+
+/**
+ * struct pvrsgx_vm - A VM context: one page directory
+ *
+ * Maps only the client heaps, plus (later) the kernel-owned structures the TA
+ * and 3D read under it. Never the kernel code or data heaps: that is what keeps
+ * a client's tasks away from the firmware's memory. A single-context core is
+ * the exception: there every VM shares the firmware heaps' page tables.
+ */
+struct pvrsgx_vm {
+	/** @gpuvm: The GPU VA manager. */
+	struct drm_gpuvm gpuvm;
+	/** @sdev: The device. */
+	struct pvrsgx_device *sdev;
+	/** @mmu: Its page directory and tables. */
+	struct pvrsgx_mmu *mmu;
+	/** @params: 3D parameters addresses of this VM's kernel objects (pvrsgx_kobj.c). */
+	struct drm_mm params;
+	/** @pb: The device's parameter buffer, mapped here by the first render context. */
+	struct pvrsgx_pb *pb;
+	/** @pb_nodes: @pb's PB and free list in @params. */
+	struct drm_mm_node pb_nodes[2];
+	/** @lock: Serializes binds, the teardown and @pb's mapping. */
+	struct mutex lock;
+	/** @ref: Held by the handle, its contexts and their jobs. */
+	struct kref ref;
+	/** @users: The handle and the live contexts; at zero its mappings go. */
+	atomic_t users;
+	/** @ran_gen: &pvrsgx_device.fw_gen + 1 at its last job sent; 0 before any. */
+	u32 ran_gen;
+	/** @pd_slot: Its slot in the microkernel's PD table (single-context cores), or 0. */
+	int pd_slot;
+};
+
+struct pvrsgx_vm *pvrsgx_vm_create(struct pvrsgx_device *sdev);
+u32 pvrsgx_vm_pd_word(const struct pvrsgx_vm *vm);
+struct pvrsgx_vm *pvrsgx_vm_get(struct pvrsgx_vm *vm);
+void pvrsgx_vm_put(struct pvrsgx_vm *vm);
+void pvrsgx_vm_user_get(struct pvrsgx_vm *vm);
+void pvrsgx_vm_user_put(struct pvrsgx_vm *vm);
+struct dma_resv *pvrsgx_vm_resv(struct pvrsgx_vm *vm);
+struct pvrsgx_vm *pvrsgx_vm_lookup(struct pvrsgx_file *sfile, u32 handle);
+void pvrsgx_vm_destroy_all(struct pvrsgx_file *sfile);
+int pvrsgx_heap_find(const struct pvrsgx_device *sdev, u64 addr, u64 size);
+
+int pvrsgx_ioctl_create_vm_context(struct drm_device *drm, void *data, struct drm_file *file);
+int pvrsgx_ioctl_destroy_vm_context(struct drm_device *drm, void *data, struct drm_file *file);
+int pvrsgx_ioctl_vm_bind(struct drm_device *drm, void *data, struct drm_file *file);
+int pvrsgx_query_heap_info(struct pvrsgx_device *sdev, void __user *ptr, u32 *size);
+
+#endif /* PVRSGX_VM_H */

-- 
2.50.1

[PATCH RFC 08/13] drm/powervr_sgx: Add contexts, the scheduler and SUBMIT_JOBS

[Joseph Telaak]

- Render contexts, each a drm_sched entity with its priority, on one
  scheduler that runs one job at a time.
- SUBMIT_JOBS with syncobj waits and signals (binary and timeline).
  Sync is explicit: the out-fence goes on the VM's reservation as
  bookkeeping only, and other drivers sync through the dma-buf
  sync-file ioctls.
- Hardware fences signaled in order from the microkernel's interrupt,
  on a reclaim-safe workqueue. A microkernel that doesn't count
  finished jobs is polled instead.
- A job that runs past its timeout resets the GPU. Its context is
  guilty, the others innocent; the guilty context's queued jobs fail.
  If the GPU can't be restarted, the driver sends a wedged uevent
  (rebind) and fails every job with -EIO.
- Tracepoints for kicks, interrupts and completions.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/powervr_sgx/pvrsgx_job.c   | 1028 ++++++++++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_job.h   |  143 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_trace.c |    5 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_trace.h |  134 ++++
 4 files changed, 1310 insertions(+)

diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_job.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_job.c
new file mode 100644
index 000000000..bc1e8ea28
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_job.c
@@ -0,0 +1,1028 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Contexts, jobs and the scheduler. One drm_sched scheduler with a credit limit
+ * of one: a single job on the GPU at a time. Each job becomes one kernel CCB
+ * command and is complete when the microkernel has finished it. A microkernel
+ * that counts finished jobs (PVRSGX_UKFW_DONE_SEQ) interrupts for each one, and
+ * a slow poll is only a backstop; with an older one, completion is also polled
+ * every jiffy while jobs are out.
+ */
+
+#include <drm/drm_auth.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_exec.h>
+#include <drm/drm_file.h>
+#include <drm/drm_gem.h>
+#include <drm/drm_gem_shmem_helper.h>
+#include <drm/drm_print.h>
+#include <drm/drm_syncobj.h>
+
+#include <linux/capability.h>
+#include <linux/dma-fence-chain.h>
+#include <linux/dma-resv.h>
+#include <linux/ktime.h>
+#include <linux/module.h>
+#include <linux/sched/mm.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <linux/workqueue.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_ccb.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_pm.h"
+#include "pvrsgx_drv.h"
+#include "pvrsgx_dump.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_prime.h"
+#include "pvrsgx_kobj.h"
+#include "pvrsgx_trace.h"
+#include "pvrsgx_validate.h"
+#include "pvrsgx_vm.h"
+
+#define PVRSGX_MAX_JOBS		64
+#define PVRSGX_MAX_SYNC_OPS	64
+static unsigned int job_timeout_ms = 2000;
+module_param(job_timeout_ms, uint, 0444);
+MODULE_PARM_DESC(job_timeout_ms, "How long a job may run before the GPU is reset (default 2000 ms)");
+
+/*
+ * Two jobs on the GPU at once, so one render's TA runs while another's 3D
+ * finishes. Off until proven: it needs the microkernel to take a TA while a 3D
+ * is in flight. A reset then cancels both, and both contexts are refused.
+ */
+
+static struct pvrsgx_job *to_pvrsgx_job(struct drm_sched_job *sched_job)
+{
+	return container_of(sched_job, struct pvrsgx_job, base);
+}
+
+static const char *fence_driver_name(struct dma_fence *fence)
+{
+	return "powervr_sgx";
+}
+
+static const char *fence_timeline_name(struct dma_fence *fence)
+{
+	return "sgx";
+}
+
+static const struct dma_fence_ops pvrsgx_fence_ops = {
+	.get_driver_name = fence_driver_name,
+	.get_timeline_name = fence_timeline_name,
+};
+
+struct pvrsgx_usage *pvrsgx_usage_new(void)
+{
+	struct pvrsgx_usage *u = kzalloc_obj(*u);
+
+	if (u)
+		kref_init(&u->ref);
+	return u;
+}
+
+struct pvrsgx_usage *pvrsgx_usage_get(struct pvrsgx_usage *u)
+{
+	kref_get(&u->ref);
+	return u;
+}
+
+static void usage_release(struct kref *ref)
+{
+	kfree(container_of(ref, struct pvrsgx_usage, ref));
+}
+
+void pvrsgx_usage_put(struct pvrsgx_usage *u)
+{
+	if (u)
+		kref_put(&u->ref, usage_release);
+}
+
+static void context_release(struct kref *ref)
+{
+	struct pvrsgx_context *ctx = container_of(ref, struct pvrsgx_context, ref);
+	struct pvrsgx_device *sdev = ctx->sdev;
+
+	pvrsgx_render_ctx_fini(sdev, ctx);
+	pvrsgx_transfer_ctx_fini(sdev, ctx);
+	/*
+	 * The next command flushes the GPU's data caches, or a later client at
+	 * the same addresses reads this one's.
+	 */
+	mutex_lock(&sdev->ccb_lock);
+	sdev->fw.cache_control |= PVRSGX_CC_INVAL_DATA;
+	mutex_unlock(&sdev->ccb_lock);
+	pvrsgx_vm_put(ctx->vm);
+	pvrsgx_usage_put(ctx->usage);
+	kfree(ctx);
+}
+
+void pvrsgx_context_put(struct pvrsgx_context *ctx)
+{
+	kref_put(&ctx->ref, context_release);
+}
+
+static void context_put(struct pvrsgx_context *ctx)
+{
+	pvrsgx_context_put(ctx);
+}
+
+/*
+ * How many of the oldest jobs on the GPU the microkernel says it has finished,
+ * when it counts them (PVRSGX_UKFW_DONE_SEQ); otherwise -1. Jobs finish in
+ * order, and POWER and cleanup commands are never on the active list.
+ */
+static s64 jobs_finished(struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_fw_ta3d_ctl *t = sdev->fw.cpu[PVRSGX_FW_TA3DCTL];
+
+	if (!(sdev->fw.features & PVRSGX_UKFW_DONE_SEQ))
+		return -1;
+	return (u32)(READ_ONCE(t->done_seq) - sdev->done_seq);
+}
+
+static bool job_done(struct pvrsgx_job *job, u64 consumed)
+{
+	if (job->done_cpu)
+		return (s32)(READ_ONCE(*job->done_cpu) - job->done_val) >= 0;
+	return job->ccb_seq <= consumed;
+}
+
+/* Whether the oldest job on the GPU is done; for the hard IRQ handler. */
+bool pvrsgx_job_oldest_done(struct pvrsgx_device *sdev)
+{
+	struct pvrsgx_job *job;
+	unsigned long flags;
+	bool done;
+
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	job = list_first_entry_or_null(&sdev->active, struct pvrsgx_job, node);
+	done = job && (sdev->fw.features & PVRSGX_UKFW_DONE_SEQ ? jobs_finished(sdev) > 0 :
+		       job_done(job, pvrsgx_ccb_consumed(sdev)));
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	return done;
+}
+
+/*
+ * A page fault stalls the faulting requestor, and the SGX545 raises no host
+ * event for it: check the BIF here, and end the job at once rather than at its
+ * timeout. Only while jobs are out, so the core is powered.
+ */
+static bool pvrsgx_job_fault_check(struct pvrsgx_device *sdev)
+{
+	bool fault;
+	int idx;
+
+	if (READ_ONCE(sdev->faulted) || READ_ONCE(sdev->lost) || !READ_ONCE(sdev->powered) ||
+	    !drm_dev_enter(&sdev->base, &idx))
+		return false;
+	fault = sgx_rd(sdev, SGX_CR_BIF_INT_STAT) & sdev->cr->bif_int_stat_fault_req;
+	drm_dev_exit(idx);
+	if (!fault)
+		return false;
+	WRITE_ONCE(sdev->faulted, true);
+	drm_sched_fault(&sdev->sched);
+	return true;
+}
+
+/* Signal every job the firmware has finished. Runs in the IRQ thread and the poll work. */
+void pvrsgx_job_process_done(struct pvrsgx_device *sdev)
+{
+	struct pvrsgx_job *job, *tmp;
+	unsigned long flags;
+	bool more, reap = false;
+	LIST_HEAD(finished);
+	bool cookie;
+	u64 consumed;
+	s64 n;
+
+	mutex_lock(&sdev->done_lock);
+	/* Under the lock a reset fails the old jobs under: no count from before it. */
+	consumed = pvrsgx_ccb_consumed(sdev);
+	cookie = dma_fence_begin_signalling();
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	n = jobs_finished(sdev);
+	list_for_each_entry_safe(job, tmp, &sdev->active, node) {
+		if (n >= 0 ? !n : !job_done(job, consumed))
+			break;
+		if (n > 0) {
+			n--;
+			sdev->done_seq++;
+		}
+		list_move_tail(&job->node, &finished);
+		if (job->type == DRM_POWERVR_SGX_JOB_TYPE_RENDER) {
+			sdev->renders_done++;
+			reap = true;
+		}
+	}
+	more = !list_empty(&sdev->active);
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+
+	list_for_each_entry(job, &finished, node)
+		pvrsgx_prime_flush(job, true);
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	list_for_each_entry_safe(job, tmp, &finished, node) {
+		list_del_init(&job->node);
+		if (job->type == DRM_POWERVR_SGX_JOB_TYPE_RENDER)
+			atomic64_add(ktime_get_ns() - job->start_ns, &job->ctx->usage->render_ns);
+		trace_pvrsgx_job_done(job->type, job->hw_fence->seqno);
+		dma_fence_signal_locked(job->hw_fence);
+	}
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	dma_fence_end_signalling(cookie);
+	mutex_unlock(&sdev->done_lock);
+
+	if (reap || !list_empty_careful(&sdev->inval_waits))
+		queue_work(sdev->free_wq, &sdev->reap_work);
+	if (more && pvrsgx_job_fault_check(sdev))
+		return;
+	/* With a count the interrupt does it, and the poll is only a backstop. */
+	if (more)
+		mod_delayed_work(sdev->wq, &sdev->poll_work,
+				 sdev->fw.features & PVRSGX_UKFW_DONE_SEQ ? HZ / 10 : 1);
+}
+
+static void reap_work_fn(struct work_struct *work)
+{
+	struct pvrsgx_device *sdev = container_of(work, struct pvrsgx_device, reap_work);
+	unsigned long flags;
+	u64 done;
+
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	done = sdev->renders_done;
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	pvrsgx_kobj_reap(sdev, done);
+	pvrsgx_ccb_inval_reap(sdev, false);
+}
+
+static void poll_work_fn(struct work_struct *work)
+{
+	struct pvrsgx_device *sdev = container_of(to_delayed_work(work), struct pvrsgx_device,
+						  poll_work);
+
+	pvrsgx_job_process_done(sdev);
+}
+
+static struct dma_fence *pvrsgx_run_job(struct drm_sched_job *sched_job)
+{
+	struct pvrsgx_job *job = to_pvrsgx_job(sched_job);
+	struct pvrsgx_device *sdev = job->ctx->sdev;
+	u32 data[6] = {};
+	struct dma_fence *fence;
+	unsigned long flags;
+	bool cookie;
+	int err, idx;
+
+	if (sched_job->s_fence->finished.error)
+		return NULL;
+	/* A reset context's queued jobs build on work that never finished. */
+	if (READ_ONCE(job->ctx->state) != DRM_POWERVR_SGX_CONTEXT_STATE_OK)
+		return ERR_PTR(-ECANCELED);
+
+	/* Allocated at submit: nothing here may wait for memory (dma-fence signalling). */
+	fence = job->hw_fence_mem;
+	job->hw_fence_mem = NULL;
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	dma_fence_init(fence, &pvrsgx_fence_ops, &sdev->fence_lock, sdev->fence_context,
+		       ++sdev->fence_seqno);
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	job->hw_fence = dma_fence_get(fence);
+
+	/* After unbind the doorbell is gone (pvrsgx_job_unplug). */
+	if (!drm_dev_enter(&sdev->base, &idx)) {
+		err = -ENODEV;
+		goto fail;
+	}
+	/* The job's runtime-PM reference, taken at submit, keeps the GPU up. */
+	if (sdev->lost) {
+		err = -EIO;
+		goto fail_exit;
+	}
+
+	pvrsgx_prime_flush(job, false);
+	WRITE_ONCE(job->ctx->vm->ran_gen, sdev->fw_gen + 1);
+	if (job->type == DRM_POWERVR_SGX_JOB_TYPE_RENDER) {
+		mutex_lock(&sdev->kobj_lock);
+		sdev->renders_submitted++;
+		mutex_unlock(&sdev->kobj_lock);
+		err = 0;
+		for (u32 k = 0; k < job->render.nkicks && !err; k++) {
+			pvrsgx_render_job_build(job, k, data);
+			err = pvrsgx_ccb_send(sdev, PVRSGX_KCMD_TA, data, &job->ccb_seq);
+		}
+	} else if (job->type == DRM_POWERVR_SGX_JOB_TYPE_TRANSFER) {
+		pvrsgx_transfer_job_build(job, data);
+		err = pvrsgx_ccb_send(sdev, PVRSGX_KCMD_TRANSFER, data, &job->ccb_seq);
+	} else {
+		/* A NULL job is a no-op command: done when the microkernel has taken it. */
+		err = pvrsgx_ccb_send(sdev, PVRSGX_KCMD_PROCESS_QUEUES, data, &job->ccb_seq);
+	}
+	if (err) {
+		drm_err_ratelimited(&sdev->base, "job not sent to the microkernel (%d)\n", err);
+		goto fail_exit;
+	}
+
+	job->start_ns = ktime_get_ns();
+	trace_pvrsgx_kick(job->type, job->ccb_seq, fence->seqno);
+	dma_fence_put(sdev->last_fence);
+	sdev->last_fence = dma_fence_get(fence);
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	list_add_tail(&job->node, &sdev->active);
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	mod_delayed_work(sdev->wq, &sdev->poll_work, 0);
+	drm_dev_exit(idx);
+	return fence;
+
+fail_exit:
+	drm_dev_exit(idx);
+fail:
+	cookie = dma_fence_begin_signalling();
+	dma_fence_set_error(fence, err);
+	dma_fence_signal(fence);
+	dma_fence_end_signalling(cookie);
+	return fence;
+}
+
+/*
+ * A job ran too long. Blame its context, fail what was on the GPU, and start
+ * the microkernel again from a clean control block and an empty CCB, rather
+ * than rescan the old one.
+ */
+static enum drm_gpu_sched_stat pvrsgx_timedout_job(struct drm_sched_job *sched_job)
+{
+	struct pvrsgx_job *bad = to_pvrsgx_job(sched_job), *job, *tmp;
+	struct pvrsgx_device *sdev = bad->ctx->sdev;
+	bool fault = READ_ONCE(sdev->faulted), lost, cookie;
+	unsigned long flags;
+	int idx;
+
+	drm_sched_stop(&sdev->sched, sched_job);
+	/* Every fence on the GPU and in the queue waits on what follows. */
+	cookie = dma_fence_begin_signalling();
+	/* After unbind (pvrsgx_job_unplug) the hardware is gone: fail, touch nothing. */
+	mutex_lock(&sdev->reset_lock);
+	lost = READ_ONCE(sdev->lost) || !drm_dev_enter(&sdev->base, &idx);
+	if (lost)
+		goto fail;
+	if (fault)
+		drm_err(&sdev->base, "job page fault at 0x%08x (BIF status 0x%08x); resetting\n",
+			sgx_rd(sdev, SGX_CR_BIF_FAULT) & sdev->cr->bif_fault_addr,
+			sgx_rd(sdev, SGX_CR_BIF_INT_STAT));
+	else
+		drm_err(&sdev->base, "job timed out (ccb %llu, consumed %llu); resetting\n",
+			bad->ccb_seq, pvrsgx_ccb_consumed(sdev));
+	WRITE_ONCE(bad->ctx->state, DRM_POWERVR_SGX_CONTEXT_STATE_GUILTY);
+	pvrsgx_coredump(sdev, bad);
+	/* Stopped before any fence signals: userspace may free the jobs' memory then. */
+	pvrsgx_fw_halt(sdev);
+
+fail:
+	mutex_lock(&sdev->done_lock);
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	list_for_each_entry_safe(job, tmp, &sdev->active, node) {
+		list_del_init(&job->node);
+		if (job->ctx != bad->ctx)
+			WRITE_ONCE(job->ctx->state, DRM_POWERVR_SGX_CONTEXT_STATE_INNOCENT);
+		dma_fence_set_error(job->hw_fence, lost ? -ENODEV : job != bad ? -ECANCELED :
+				    fault ? -EFAULT : -ETIME);
+		dma_fence_signal_locked(job->hw_fence);
+	}
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	mutex_unlock(&sdev->done_lock);
+	if (lost)
+		goto out;
+
+	/*
+	 * No power-off handshake with a hung microkernel, which cannot answer:
+	 * straight into the full start (clocks, scripts, a reset that drains
+	 * pending faults, the first kick).
+	 */
+	sdev->reset_count++;
+	sdev->fw_running = false;
+	/* The restart's reset clears the BIF's fault state. */
+	WRITE_ONCE(sdev->faulted, false);
+	if (pvrsgx_fw_start(sdev)) {
+		drm_err(&sdev->base, "microkernel did not restart: device lost\n");
+		/* Lost: the cores stay in reset, and run_job fails every job. */
+		pvrsgx_fw_halt(sdev);
+		WRITE_ONCE(sdev->lost, true);
+		/* DMA is off and the fences are signaled: tell userspace to rebind. */
+		drm_dev_wedged_event(&sdev->base, DRM_WEDGE_RECOVERY_REBIND, NULL);
+	}
+	drm_dev_exit(idx);
+out:
+	mutex_unlock(&sdev->reset_lock);
+	dma_fence_end_signalling(cookie);
+	drm_sched_start(&sdev->sched, 0);
+	return DRM_GPU_SCHED_STAT_RESET;
+}
+
+/* The puts that can reach a VM's release, which takes reservations. */
+static void free_job_work(struct work_struct *work)
+{
+	struct pvrsgx_job *job = container_of(work, struct pvrsgx_job, free_work);
+
+	pvrsgx_prime_put(job);
+	pvrsgx_render_job_fini(job);
+	context_put(job->ctx);
+	kfree(job);
+}
+
+static void pvrsgx_free_job(struct drm_sched_job *sched_job)
+{
+	struct pvrsgx_job *job = to_pvrsgx_job(sched_job);
+	struct pvrsgx_device *sdev = job->ctx->sdev;
+
+	/* Taken at submit, for this job: the GPU may suspend once none are left. */
+	pvrsgx_power_put(sdev, 1);
+
+	drm_sched_job_cleanup(sched_job);
+	dma_fence_put(job->hw_fence);
+	kfree(job->hw_fence_mem);	/* never run */
+	INIT_WORK(&job->free_work, free_job_work);
+	queue_work(sdev->free_wq, &job->free_work);
+}
+
+static const struct drm_sched_backend_ops pvrsgx_sched_ops = {
+	.run_job = pvrsgx_run_job,
+	.timedout_job = pvrsgx_timedout_job,
+	.free_job = pvrsgx_free_job,
+};
+
+/*
+ * Teach lockdep that the locks run_job, the completion and the timeout take
+ * are taken while fences are pending, which reclaim can wait for: an
+ * allocation that can reclaim, made under any of them, is then reported at
+ * once instead of when memory runs low.
+ */
+static void lockdep_prime(struct pvrsgx_device *sdev)
+{
+	if (!IS_ENABLED(CONFIG_LOCKDEP))
+		return;
+	fs_reclaim_acquire(GFP_KERNEL);
+	might_lock(&sdev->reset_lock);
+	might_lock(&sdev->done_lock);
+	might_lock(&sdev->ccb_lock);
+	might_lock(&sdev->kobj_lock);
+	fs_reclaim_release(GFP_KERNEL);
+}
+
+int pvrsgx_job_init(struct pvrsgx_device *sdev)
+{
+	const struct drm_sched_init_args args = {
+		.ops = &pvrsgx_sched_ops,
+		.num_rqs = DRM_SCHED_PRIORITY_COUNT,
+		.credit_limit = 1,
+		.timeout = msecs_to_jiffies(job_timeout_ms),
+		.name = "powervr_sgx",
+		.dev = sdev->base.dev,
+	};
+
+	int err;
+
+	spin_lock_init(&sdev->fence_lock);
+	mutex_init(&sdev->reset_lock);
+	INIT_LIST_HEAD(&sdev->active);
+	INIT_DELAYED_WORK(&sdev->poll_work, poll_work_fn);
+	INIT_WORK(&sdev->reap_work, reap_work_fn);
+	sdev->wq = alloc_workqueue("powervr_sgx", WQ_MEM_RECLAIM | WQ_UNBOUND, 0);
+	if (!sdev->wq)
+		return -ENOMEM;
+	sdev->free_wq = alloc_workqueue("powervr_sgx_free", WQ_UNBOUND, 0);
+	if (!sdev->free_wq) {
+		destroy_workqueue(sdev->wq);
+		return -ENOMEM;
+	}
+	lockdep_prime(sdev);
+	pvrsgx_coredump_init(sdev);
+	sdev->fence_context = dma_fence_context_alloc(1);
+	err = drm_sched_init(&sdev->sched, &args);
+	if (err) {
+		pvrsgx_coredump_fini(sdev);
+		destroy_workqueue(sdev->free_wq);
+		destroy_workqueue(sdev->wq);
+	}
+	return err;
+}
+
+/*
+ * Unbind. Nothing reaches the GPU again: the firmware stops, the jobs on it
+ * fail with -ENODEV, and run_job fails the queued ones (-EIO, @lost) without
+ * touching the hardware.
+ */
+void pvrsgx_job_unplug(struct pvrsgx_device *sdev)
+{
+	struct pvrsgx_job *job, *tmp;
+	unsigned long flags;
+	bool cookie;
+
+	drm_sched_wqueue_stop(&sdev->sched);
+	/* A reset under way finishes first; one after this sees @lost. */
+	mutex_lock(&sdev->reset_lock);
+	WRITE_ONCE(sdev->lost, true);
+	pvrsgx_fw_stop(sdev);
+	mutex_unlock(&sdev->reset_lock);
+	cancel_delayed_work_sync(&sdev->poll_work);
+	/* Stopped: nothing translates any more, so unmapped memory can go. */
+	pvrsgx_ccb_inval_reap(sdev, true);
+
+	mutex_lock(&sdev->done_lock);
+	cookie = dma_fence_begin_signalling();
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	list_for_each_entry_safe(job, tmp, &sdev->active, node) {
+		list_del_init(&job->node);
+		dma_fence_set_error(job->hw_fence, -ENODEV);
+		dma_fence_signal_locked(job->hw_fence);
+	}
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	dma_fence_end_signalling(cookie);
+	mutex_unlock(&sdev->done_lock);
+	drm_sched_wqueue_start(&sdev->sched);
+}
+
+/*
+ * System sleep: no new job reaches the GPU, and the jobs on it finish, or the
+ * timeout's reset ends them, within a job timeout. -EBUSY aborts the suspend.
+ */
+int pvrsgx_job_quiesce(struct pvrsgx_device *sdev)
+{
+	long left = msecs_to_jiffies(job_timeout_ms) + HZ;
+
+	/* A reset restarts submission; stop it again after one. */
+	for (int i = 0; i < 2; i++) {
+		struct dma_fence *last;
+
+		drm_sched_wqueue_stop(&sdev->sched);
+		/* Set only by run_job, which is stopped. Jobs finish in order. */
+		last = sdev->last_fence;
+		if (!last || dma_fence_is_signaled(last)) {
+			flush_work(&sdev->reap_work);
+			return 0;
+		}
+		left = dma_fence_wait_timeout(last, false, left);
+		if (left <= 0)
+			break;
+	}
+	drm_err(&sdev->base, "GPU still busy: not suspending\n");
+	drm_sched_wqueue_start(&sdev->sched);
+	return -EBUSY;
+}
+
+void pvrsgx_job_unquiesce(struct pvrsgx_device *sdev)
+{
+	drm_sched_wqueue_start(&sdev->sched);
+}
+
+void pvrsgx_job_fini(struct pvrsgx_device *sdev)
+{
+	drm_sched_fini(&sdev->sched);
+	cancel_delayed_work_sync(&sdev->poll_work);
+	cancel_work_sync(&sdev->reap_work);
+	/* Drains the jobs' final puts, which drm_sched_fini may have queued. */
+	destroy_workqueue(sdev->free_wq);
+	destroy_workqueue(sdev->wq);
+	pvrsgx_coredump_fini(sdev);
+	dma_fence_put(sdev->last_fence);
+	sdev->last_fence = NULL;
+}
+
+static enum drm_sched_priority sched_priority(s32 p)
+{
+	switch (p) {
+	case DRM_POWERVR_SGX_CTX_PRIORITY_LOW:
+		return DRM_SCHED_PRIORITY_LOW;
+	case DRM_POWERVR_SGX_CTX_PRIORITY_HIGH:
+		return DRM_SCHED_PRIORITY_HIGH;
+	default:
+		return DRM_SCHED_PRIORITY_NORMAL;
+	}
+}
+
+int pvrsgx_ioctl_create_context(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_create_context_args *args = data;
+	struct pvrsgx_device *sdev = to_pvrsgx_device(drm);
+	struct pvrsgx_file *sfile = to_pvrsgx_file(file);
+	struct drm_gpu_scheduler *sched = &sdev->sched;
+	struct pvrsgx_context *ctx;
+	int err;
+
+	if (args->flags)
+		return -EINVAL;
+	if (args->type != DRM_POWERVR_SGX_CTX_TYPE_RENDER &&
+	    args->type != DRM_POWERVR_SGX_CTX_TYPE_TRANSFER)
+		return -EINVAL;
+	if (args->priority != DRM_POWERVR_SGX_CTX_PRIORITY_LOW &&
+	    args->priority != DRM_POWERVR_SGX_CTX_PRIORITY_NORMAL &&
+	    args->priority != DRM_POWERVR_SGX_CTX_PRIORITY_HIGH)
+		return -EINVAL;
+	if (args->priority == DRM_POWERVR_SGX_CTX_PRIORITY_HIGH &&
+	    !capable(CAP_SYS_NICE) && !drm_is_current_master(file))
+		return -EACCES;
+	if (args->type == DRM_POWERVR_SGX_CTX_TYPE_TRANSFER && args->param_buffer_pages)
+		return -EINVAL;
+	if (args->param_buffer_pages &&
+	    (args->param_buffer_pages < 0x100 ||
+	     args->param_buffer_pages > pvrsgx_heaps(to_pvrsgx_device(drm))->pb_max_pages))
+		return -EINVAL;
+
+	ctx = kzalloc_obj(*ctx);
+	if (!ctx)
+		return -ENOMEM;
+	ctx->vm = pvrsgx_vm_lookup(sfile, args->vm_context_handle);
+	if (!ctx->vm) {
+		kfree(ctx);
+		return -ENOENT;
+	}
+	ctx->sdev = sdev;
+	ctx->type = args->type;
+	ctx->usage = pvrsgx_usage_get(sfile->usage);
+	kref_init(&ctx->ref);
+	err = ctx->type == DRM_POWERVR_SGX_CTX_TYPE_RENDER ?
+	      pvrsgx_render_ctx_init(sdev, ctx, args->param_buffer_pages) :
+	      pvrsgx_transfer_ctx_init(sdev, ctx);
+	if (err)
+		goto err_put;
+	err = drm_sched_entity_init(&ctx->entity, sched_priority(args->priority), &sched, 1, NULL);
+	if (err)
+		goto err_put;
+	pvrsgx_vm_user_get(ctx->vm);
+	err = xa_alloc(&sfile->contexts, &args->handle, ctx,
+		       XA_LIMIT(1, PVRSGX_MAX_FILE_CONTEXTS), GFP_KERNEL);
+	if (err) {
+		err = err == -EBUSY ? -ENOSPC : err;
+		goto err_entity;
+	}
+	return 0;
+
+err_entity:
+	drm_sched_entity_destroy(&ctx->entity);
+	pvrsgx_vm_user_put(ctx->vm);
+err_put:
+	context_put(ctx);
+	return err;
+}
+
+static void context_destroy(struct pvrsgx_context *ctx)
+{
+	/* Its queued jobs are on the GPU or cancelled before its VM's mappings may go. */
+	drm_sched_entity_destroy(&ctx->entity);
+	pvrsgx_vm_user_put(ctx->vm);
+	context_put(ctx);
+}
+
+void pvrsgx_context_destroy_all(struct pvrsgx_file *sfile)
+{
+	struct pvrsgx_context *ctx;
+	unsigned long handle;
+
+	xa_for_each(&sfile->contexts, handle, ctx) {
+		xa_erase(&sfile->contexts, handle);
+		context_destroy(ctx);
+	}
+}
+
+int pvrsgx_ioctl_destroy_context(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_destroy_context_args *args = data;
+	struct pvrsgx_context *ctx;
+
+	if (args->_padding_4)
+		return -EINVAL;
+	ctx = xa_erase(&to_pvrsgx_file(file)->contexts, args->handle);
+	if (!ctx)
+		return -ENOENT;
+	context_destroy(ctx);
+	return 0;
+}
+
+struct pvrsgx_context *pvrsgx_context_lookup(struct pvrsgx_file *sfile, u32 handle)
+{
+	struct pvrsgx_context *ctx;
+
+	xa_lock(&sfile->contexts);
+	ctx = xa_load(&sfile->contexts, handle);
+	if (ctx)
+		kref_get(&ctx->ref);
+	xa_unlock(&sfile->contexts);
+	return ctx;
+}
+
+int pvrsgx_ioctl_get_context_state(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_get_context_state_args *args = data;
+	struct pvrsgx_context *ctx;
+
+	if (args->_padding_c)
+		return -EINVAL;
+	ctx = pvrsgx_context_lookup(to_pvrsgx_file(file), args->handle);
+	if (!ctx)
+		return -ENOENT;
+	args->state = READ_ONCE(ctx->state);
+	args->reset_count = to_pvrsgx_device(drm)->reset_count;
+	context_put(ctx);
+	return 0;
+}
+
+/* What one job needs from its submission, held until it is pushed. */
+struct job_args {
+	struct pvrsgx_job *job;
+	struct pvrsgx_signal *signals;
+	u32 nsignals;
+	/* Earlier jobs of this submit whose signals this job waits on. */
+	u32 *intra;
+	u32 nintra;
+	bool inited;
+	/* Once pushed: the job's finished fence, for later jobs' waits. */
+	struct dma_fence *done;
+};
+
+static void job_args_fini(struct job_args *ja)
+{
+	for (u32 i = 0; i < ja->nsignals; i++) {
+		if (ja->signals[i].syncobj)
+			drm_syncobj_put(ja->signals[i].syncobj);
+		dma_fence_chain_free(ja->signals[i].chain);
+	}
+	kvfree(ja->signals);
+	kvfree(ja->intra);
+	dma_fence_put(ja->done);
+	if (ja->job) {
+		if (ja->inited)
+			drm_sched_job_cleanup(&ja->job->base);
+		pvrsgx_prime_put(ja->job);
+		pvrsgx_render_job_fini(ja->job);
+		context_put(ja->job->ctx);
+		kfree(ja->job->hw_fence_mem);
+		kfree(ja->job);
+	}
+}
+
+/* The latest earlier job of this submit that signals handle at point, or -1. */
+static int intra_signaller(const struct job_args *prev, u32 nprev, u32 handle, u64 point)
+{
+	for (int k = nprev - 1; k >= 0; k--) {
+		for (u32 s = 0; s < prev[k].nsignals; s++) {
+			if (prev[k].signals[s].handle == handle &&
+			    prev[k].signals[s].point == point)
+				return k;
+		}
+	}
+	return -1;
+}
+
+static int job_sync_ops(struct drm_file *file, struct job_args *ja,
+			const struct drm_powervr_sgx_obj_array *arr,
+			const struct job_args *prev, u32 nprev)
+{
+	struct drm_powervr_sgx_sync_op *ops;
+	int err = 0;
+
+	ops = pvrsgx_obj_array_get(arr, sizeof(*ops), PVRSGX_MAX_SYNC_OPS);
+	if (IS_ERR_OR_NULL(ops))
+		return PTR_ERR_OR_ZERO(ops);
+
+	ja->signals = kvcalloc(arr->count, sizeof(*ja->signals), GFP_KERNEL);
+	ja->intra = kvcalloc(arr->count, sizeof(*ja->intra), GFP_KERNEL);
+	if (!ja->signals || !ja->intra) {
+		err = -ENOMEM;
+		goto out;
+	}
+	for (u32 i = 0; i < arr->count && !err; i++) {
+		u32 type = ops[i].flags & DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_MASK;
+		bool timeline = type == DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_TIMELINE_SYNCOBJ;
+		struct dma_fence *fence;
+
+		if ((ops[i].flags & ~PVRSGX_SYNC_OP_FLAGS_MASK) ||
+		    (type != DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_SYNCOBJ && !timeline) ||
+		    (!timeline && ops[i].value)) {
+			err = -EINVAL;
+			break;
+		}
+
+		if (!(ops[i].flags & DRM_POWERVR_SGX_SYNC_OP_FLAG_SIGNAL)) {
+			int k = intra_signaller(prev, nprev, ops[i].handle, ops[i].value);
+
+			/* Its fence exists once that job is armed: added then. */
+			if (k >= 0) {
+				ja->intra[ja->nintra++] = k;
+				continue;
+			}
+			err = drm_syncobj_find_fence(file, ops[i].handle, ops[i].value, 0, &fence);
+			if (!err)
+				err = drm_sched_job_add_dependency(&ja->job->base, fence);
+			continue;
+		}
+
+		ja->signals[ja->nsignals].syncobj = drm_syncobj_find(file, ops[i].handle);
+		if (!ja->signals[ja->nsignals].syncobj) {
+			err = -ENOENT;
+			break;
+		}
+		ja->signals[ja->nsignals].handle = ops[i].handle;
+		ja->signals[ja->nsignals].point = ops[i].value;
+		if (timeline) {
+			ja->signals[ja->nsignals].chain = dma_fence_chain_alloc();
+			if (!ja->signals[ja->nsignals].chain)
+				err = -ENOMEM;
+		}
+		ja->nsignals++;
+	}
+out:
+	kvfree(ops);
+	return err;
+}
+
+static int job_create(struct drm_file *file, const struct drm_powervr_sgx_job *in,
+		      struct job_args *ja, const struct job_args *prev, u32 nprev)
+{
+	struct pvrsgx_file *sfile = to_pvrsgx_file(file);
+	struct pvrsgx_device *sdev = sfile->sdev;
+	struct pvrsgx_context *ctx;
+	struct pvrsgx_job *job;
+	int err;
+
+	if (in->_padding_c || in->_padding_4c)
+		return -EINVAL;
+
+	switch (in->type) {
+	case DRM_POWERVR_SGX_JOB_TYPE_NULL:
+		if (in->flags || in->cmd || in->ta_kicks.count ||
+		    in->rt_dataset_handle)
+			return -EINVAL;
+		break;
+	case DRM_POWERVR_SGX_JOB_TYPE_RENDER:
+	case DRM_POWERVR_SGX_JOB_TYPE_TRANSFER:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	ctx = pvrsgx_context_lookup(sfile, in->context_handle);
+	if (!ctx)
+		return -ENOENT;
+	if (READ_ONCE(ctx->state) != DRM_POWERVR_SGX_CONTEXT_STATE_OK) {
+		context_put(ctx);
+		return -ECANCELED;
+	}
+	if (sdev->lost) {
+		context_put(ctx);
+		return -EIO;
+	}
+
+	job = kzalloc_obj(*job);
+	if (!job) {
+		context_put(ctx);
+		return -ENOMEM;
+	}
+	job->ctx = ctx;
+	job->type = in->type;
+	INIT_LIST_HEAD(&job->node);
+	ja->job = job;
+	job->hw_fence_mem = kzalloc_obj(*job->hw_fence_mem);
+	if (!job->hw_fence_mem)
+		return -ENOMEM;
+
+	err = drm_sched_job_init(&job->base, &ctx->entity, 1, sfile, file->client_id);
+	if (err)
+		return err;
+	ja->inited = true;
+
+	if (job->type == DRM_POWERVR_SGX_JOB_TYPE_RENDER) {
+		if (ctx->type != DRM_POWERVR_SGX_CTX_TYPE_RENDER)
+			return -EINVAL;
+		err = pvrsgx_render_job_init(file, job, in);
+		if (err)
+			return err;
+	} else if (job->type == DRM_POWERVR_SGX_JOB_TYPE_TRANSFER) {
+		if (ctx->type != DRM_POWERVR_SGX_CTX_TYPE_TRANSFER)
+			return -EINVAL;
+		err = pvrsgx_transfer_job_init(file, job, in);
+		if (err)
+			return err;
+	}
+
+	err = job_sync_ops(file, ja, &in->sync_ops, prev, nprev);
+	if (err)
+		return err;
+	err = pvrsgx_prime_ranges(file, &in->cache_ranges, job);
+	if (err)
+		return err;
+	return pvrsgx_validate_job(sdev, job);
+}
+
+int pvrsgx_ioctl_submit_jobs(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_submit_jobs_args *args = data;
+	struct drm_powervr_sgx_job *in;
+	struct job_args *ja;
+	struct drm_exec exec;
+	u32 n = args->jobs.count;
+	u32 pushed = 0;
+	int err = 0;
+
+	args->failed_index = ~0u;
+	if (args->_padding_14 || !n)
+		return -EINVAL;
+	if (drm_dev_is_unplugged(drm))
+		return -ENODEV;
+
+	in = pvrsgx_obj_array_get(&args->jobs, sizeof(*in), PVRSGX_MAX_JOBS);
+	if (IS_ERR(in))
+		return PTR_ERR(in);
+	ja = kvcalloc(n, sizeof(*ja), GFP_KERNEL);
+	if (!ja) {
+		kvfree(in);
+		return -ENOMEM;
+	}
+
+	for (u32 i = 0; i < n; i++) {
+		err = job_create(file, &in[i], &ja[i], ja, i);
+		if (err) {
+			args->failed_index = i;
+			goto out;
+		}
+	}
+
+	/*
+	 * One runtime-PM reference per job, from here, where resuming may sleep and
+	 * no reservation is held, to free_job; never in run_job, which signals fences.
+	 */
+	for (u32 i = 0; i < n; i++) {
+		err = pvrsgx_power_get(to_pvrsgx_device(drm));
+		if (err) {
+			pvrsgx_power_put(to_pvrsgx_device(drm), i);
+			goto out;
+		}
+	}
+
+	drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT | DRM_EXEC_IGNORE_DUPLICATES, 0);
+	drm_exec_until_all_locked(&exec) {
+		for (u32 i = 0; i < n; i++) {
+			struct drm_gpuvm *gpuvm = &ja[i].job->ctx->vm->gpuvm;
+
+			err = drm_exec_prepare_obj(&exec, drm_gpuvm_resv_obj(gpuvm), n);
+			drm_exec_retry_on_contention(&exec);
+			if (err)
+				goto out_exec;
+		}
+	}
+
+	/*
+	 * All or nothing. First each job's waits on earlier jobs of the submit,
+	 * then arm it; a wait that can't be added fails the call, and the armed
+	 * jobs, never published, are cleaned up unpushed (job_args_fini). Then
+	 * publish and push every job, which can't fail. The VM's reservation,
+	 * held throughout, keeps the order.
+	 */
+	for (u32 i = 0; i < n; i++) {
+		struct pvrsgx_job *job = ja[i].job;
+
+		for (u32 d = 0; d < ja[i].nintra; d++) {
+			err = drm_sched_job_add_dependency(&job->base,
+							   dma_fence_get(ja[ja[i].intra[d]].done));
+			if (err) {
+				args->failed_index = i;
+				goto out_exec;
+			}
+		}
+		drm_sched_job_arm(&job->base);
+		ja[i].done = dma_fence_get(&job->base.s_fence->finished);
+	}
+
+	for (u32 i = 0; i < n; i++) {
+		struct pvrsgx_job *job = ja[i].job;
+		struct dma_fence *done = ja[i].done;
+
+		dma_resv_add_fence(drm_gpuvm_resv(&job->ctx->vm->gpuvm), done,
+				   DMA_RESV_USAGE_BOOKKEEP);
+		for (u32 s = 0; s < ja[i].nsignals; s++) {
+			struct pvrsgx_signal *sig = &ja[i].signals[s];
+
+			if (sig->chain) {
+				drm_syncobj_add_point(sig->syncobj, sig->chain, done, sig->point);
+				sig->chain = NULL;
+			} else {
+				drm_syncobj_replace_fence(sig->syncobj, done);
+			}
+		}
+		drm_sched_entity_push_job(&job->base);
+		ja[i].job = NULL;	/* the scheduler owns it now */
+		pushed++;
+	}
+
+out_exec:
+	drm_exec_fini(&exec);
+	/* The references of the jobs not pushed. */
+	pvrsgx_power_put(to_pvrsgx_device(drm), n - pushed);
+out:
+	for (u32 i = 0; i < n; i++)
+		job_args_fini(&ja[i]);
+	kvfree(ja);
+	kvfree(in);
+	return err;
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_job.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_job.h
new file mode 100644
index 000000000..87d3744c7
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_job.h
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_JOB_H
+#define PVRSGX_JOB_H
+
+#include <drm/gpu_scheduler.h>
+
+#include <linux/kref.h>
+#include <linux/list.h>
+#include <linux/types.h>
+
+#include "pvrsgx_render.h"
+#include "pvrsgx_transfer.h"
+
+struct drm_device;
+struct drm_file;
+struct drm_syncobj;
+struct dma_fence_chain;
+struct pvrsgx_device;
+struct pvrsgx_file;
+struct pvrsgx_vm;
+
+/**
+ * struct pvrsgx_usage - A file's GPU time, for fdinfo
+ *
+ * Its contexts hold it too: a job may finish after its file has closed.
+ */
+struct pvrsgx_usage {
+	/** @ref: Held by the file and by each of its contexts. */
+	struct kref ref;
+	/** @render_ns: Time render jobs of the file spent on the GPU. */
+	atomic64_t render_ns;
+};
+
+struct pvrsgx_usage *pvrsgx_usage_new(void);
+struct pvrsgx_usage *pvrsgx_usage_get(struct pvrsgx_usage *u);
+void pvrsgx_usage_put(struct pvrsgx_usage *u);
+
+/**
+ * struct pvrsgx_context - A queue of jobs in one VM
+ */
+struct pvrsgx_context {
+	/** @ref: Held by the handle and by each job. */
+	struct kref ref;
+	/** @sdev: The device. */
+	struct pvrsgx_device *sdev;
+	/** @vm: The VM its jobs run in. */
+	struct pvrsgx_vm *vm;
+	/** @usage: Its file's GPU time. */
+	struct pvrsgx_usage *usage;
+	/** @entity: Its scheduler queue. */
+	struct drm_sched_entity entity;
+	/** @type: enum drm_powervr_sgx_ctx_type. */
+	u32 type;
+	/** @state: enum drm_powervr_sgx_context_state. */
+	u32 state;
+	/** @render: The firmware half of a render context, else NULL. */
+	struct pvrsgx_render_ctx *render;
+	/** @transfer: The firmware half of a transfer context, else NULL. */
+	struct pvrsgx_transfer_ctx *transfer;
+};
+
+struct pvrsgx_signal {
+	u32 handle;
+	struct drm_syncobj *syncobj;
+	struct dma_fence_chain *chain;
+	u64 point;
+};
+
+/* Bytes of an import a job reads or writes: @rows rows of @size, @stride apart. */
+struct pvrsgx_cache_range {
+	struct drm_gem_object *obj;
+	u64 offset;
+	u64 size;
+	u32 stride;
+	u32 rows;
+	bool write;
+	bool import;
+};
+
+/**
+ * struct pvrsgx_job - One submitted job
+ */
+struct pvrsgx_job {
+	/** @free_work: The final puts, off the scheduler's queue. */
+	struct work_struct free_work;
+	/** @base: The scheduler job. */
+	struct drm_sched_job base;
+	/** @ctx: The context it was submitted on. */
+	struct pvrsgx_context *ctx;
+	/** @type: enum drm_powervr_sgx_job_type. */
+	u32 type;
+
+	/** @hw_fence: Signalled when the firmware has finished the job. */
+	struct dma_fence *hw_fence;
+	/** @hw_fence_mem: @hw_fence's memory from submit until run_job initializes it. */
+	struct dma_fence *hw_fence_mem;
+	/** @ccb_seq: Kernel CCB command that carries the job. */
+	/** @start_ns: When run_job sent it to the microkernel. */
+	u64 start_ns;
+	u64 ccb_seq;
+	/** @node: In pvrsgx_device.active while on the GPU. */
+	struct list_head node;
+	/**
+	 * @done_cpu: Word the firmware sets to @done_val when the job is done;
+	 * NULL for a job that is done when the CCB has taken it.
+	 */
+	u32 *done_cpu;
+	/** @done_val: Value of *@done_cpu once the job is done. */
+	u32 done_val;
+	/** @render: The render job, for a render context. */
+	struct pvrsgx_render_job render;
+	/** @transfer: The transfer job, for a transfer context. */
+	struct pvrsgx_transfer_job transfer;
+
+	/** @ranges: The bytes of imports the job reads or writes, flushed around it. */
+	struct pvrsgx_cache_range *ranges;
+	/** @nranges: Number of @ranges. */
+	u32 nranges;
+	/** @signals: Syncobjs to signal with the job's fence. */
+	struct pvrsgx_signal *signals;
+	/** @nsignals: Number of @signals. */
+	u32 nsignals;
+};
+
+int pvrsgx_job_init(struct pvrsgx_device *sdev);
+void pvrsgx_job_fini(struct pvrsgx_device *sdev);
+void pvrsgx_job_process_done(struct pvrsgx_device *sdev);
+void pvrsgx_job_unplug(struct pvrsgx_device *sdev);
+int pvrsgx_job_quiesce(struct pvrsgx_device *sdev);
+void pvrsgx_job_unquiesce(struct pvrsgx_device *sdev);
+bool pvrsgx_job_oldest_done(struct pvrsgx_device *sdev);
+void pvrsgx_context_destroy_all(struct pvrsgx_file *sfile);
+struct pvrsgx_context *pvrsgx_context_lookup(struct pvrsgx_file *sfile, u32 handle);
+void pvrsgx_context_put(struct pvrsgx_context *ctx);
+
+int pvrsgx_ioctl_create_context(struct drm_device *drm, void *data, struct drm_file *file);
+int pvrsgx_ioctl_destroy_context(struct drm_device *drm, void *data, struct drm_file *file);
+int pvrsgx_ioctl_get_context_state(struct drm_device *drm, void *data, struct drm_file *file);
+int pvrsgx_ioctl_submit_jobs(struct drm_device *drm, void *data, struct drm_file *file);
+
+#endif /* PVRSGX_JOB_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_trace.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_trace.c
new file mode 100644
index 000000000..5c70244c3
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_trace.c
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+#define CREATE_TRACE_POINTS
+#include "pvrsgx_trace.h"
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_trace.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_trace.h
new file mode 100644
index 000000000..aab9bc2f6
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_trace.h
@@ -0,0 +1,134 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM powervr_sgx
+
+#if !defined(PVRSGX_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
+#define PVRSGX_TRACE_H
+
+#include <linux/tracepoint.h>
+#include <linux/types.h>
+
+#include "pvrsgx_fwif.h"
+
+/* A command handed to the microkernel (kernel CCB, doorbell rung). */
+TRACE_EVENT(pvrsgx_kick,
+	TP_PROTO(u32 type, u64 seq, u64 fence_seqno),
+	TP_ARGS(type, seq, fence_seqno),
+	TP_STRUCT__entry(
+		__field(u32, type)
+		__field(u64, seq)
+		__field(u64, fence_seqno)
+	),
+	TP_fast_assign(
+		__entry->type = type;
+		__entry->seq = seq;
+		__entry->fence_seqno = fence_seqno;
+	),
+	TP_printk("type=%u ccb=%llu fence=%llu", __entry->type, __entry->seq,
+		  __entry->fence_seqno)
+);
+
+/* A render's last kick: the dataset and the per-frame values it was given. */
+TRACE_EVENT(pvrsgx_render_kick,
+	TP_PROTO(const struct pvrsgx_fw_ta_cmd *cmd, u32 w, u32 h, u32 rgn_next, u32 cmd_off),
+	TP_ARGS(cmd, w, h, rgn_next, cmd_off),
+	TP_STRUCT__entry(
+		__field(u32, rtdata)
+		__field(u32, w)
+		__field(u32, h)
+		__field(u32, frame)
+		__field(u32, rt_entry)
+		__field(u32, rgn_this)
+		__field(u32, rgn_next)
+		__field(u32, cmd_off)
+		__field(u32, pb_desc)
+		__field(u32, te_screen)
+		__field(u32, te_mtile0)
+		__field(u32, te_mtile1)
+		__field(u32, mt_tiles)
+		__field(u32, screen_size)
+	),
+	TP_fast_assign(
+		__entry->rtdata = cmd->rt_data;
+		__entry->w = w;
+		__entry->h = h;
+		__entry->frame = cmd->frame;
+		__entry->rt_entry = cmd->rt_data_entry;
+		__entry->rgn_this = cmd->region_headers;
+		__entry->rgn_next = rgn_next;
+		__entry->cmd_off = cmd_off;
+		__entry->pb_desc = cmd->pb_desc;
+		__entry->te_screen = cmd->te_screen;
+		__entry->te_mtile0 = cmd->te_mtile[0];
+		__entry->te_mtile1 = cmd->te_mtile[1];
+		__entry->mt_tiles = cmd->mt_tiles;
+		__entry->screen_size = cmd->screen_size;
+	),
+	TP_printk("rtdata=0x%08x %ux%u frame=%u entry=0x%08x rgn=0x%08x next=0x%08x cmd_off=0x%x pb=0x%08x te_screen=0x%08x mtile=0x%08x,0x%08x mt_tiles=0x%08x screen=0x%08x",
+		  __entry->rtdata, __entry->w, __entry->h, __entry->frame,
+		  __entry->rt_entry, __entry->rgn_this, __entry->rgn_next,
+		  __entry->cmd_off, __entry->pb_desc, __entry->te_screen,
+		  __entry->te_mtile0, __entry->te_mtile1, __entry->mt_tiles,
+		  __entry->screen_size)
+);
+
+/* The microkernel's interrupt, in the hard handler. */
+TRACE_EVENT(pvrsgx_irq,
+	TP_PROTO(u32 status),
+	TP_ARGS(status),
+	TP_STRUCT__entry(
+		__field(u32, status)
+	),
+	TP_fast_assign(
+		__entry->status = status;
+	),
+	TP_printk("status=0x%08x", __entry->status)
+);
+
+/* A job seen done, just before its hardware fence signals. */
+TRACE_EVENT(pvrsgx_job_done,
+	TP_PROTO(u32 type, u64 fence_seqno),
+	TP_ARGS(type, fence_seqno),
+	TP_STRUCT__entry(
+		__field(u32, type)
+		__field(u64, fence_seqno)
+	),
+	TP_fast_assign(
+		__entry->type = type;
+		__entry->fence_seqno = fence_seqno;
+	),
+	TP_printk("type=%u fence=%llu", __entry->type, __entry->fence_seqno)
+);
+
+/* A job's imports flushed from the CPU caches. */
+TRACE_EVENT(pvrsgx_flush_imports,
+	TP_PROTO(u64 fence_seqno, u32 nimports, u64 bytes, u64 ns, bool after),
+	TP_ARGS(fence_seqno, nimports, bytes, ns, after),
+	TP_STRUCT__entry(
+		__field(u64, fence_seqno)
+		__field(u32, nimports)
+		__field(u64, bytes)
+		__field(u64, ns)
+		__field(bool, after)
+	),
+	TP_fast_assign(
+		__entry->fence_seqno = fence_seqno;
+		__entry->nimports = nimports;
+		__entry->bytes = bytes;
+		__entry->ns = ns;
+		__entry->after = after;
+	),
+	TP_printk("fence=%llu %s imports=%u bytes=%llu ns=%llu", __entry->fence_seqno,
+		  __entry->after ? "after" : "before", __entry->nimports, __entry->bytes,
+		  __entry->ns)
+);
+
+#endif /* PVRSGX_TRACE_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE pvrsgx_trace
+#include <trace/define_trace.h>

-- 
2.50.1

[PATCH RFC 09/13] drm/powervr_sgx: Add the client command checks

[Joseph Telaak]

Userspace describes a job's work as plain structs, never as a firmware
structure: a render command and one TA-kick command per kick. Each
starts with its size and a chain of extensions
for the fields only one core has.

The kernel copies each struct as copy_struct_from_user() does: a
shorter struct reads as zeros, and non-zero bytes past what the kernel
knows are -E2BIG. It walks the extension chain (at most 16, each name
once; one this core doesn't accept is -EOPNOTSUPP) and checks what it
copied, so a racing writer can't change a checked value. Every field is
checked against the core's table: ranges, masks, alignment, and for an
address the heap it must lie in and an extent mapped in the job's VM.
Then the cross-field rules (depth and stencil with ZLS, tile
rectangles), and only then does it build the firmware command from the
checked values.

The structs, their extensions and the per-core check tables come from
one XML description. The generated tables are committed, so the build
needs no generator.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.c        | 234 ++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.h        | 132 +++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.c |  62 +++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.h |  13 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.c   | 353 ++++++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.h   |  43 +++
 6 files changed, 837 insertions(+)

diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.c
new file mode 100644
index 000000000..0da4f387c
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.c
@@ -0,0 +1,234 @@
+// SPDX-License-Identifier: MIT
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Client commands: the copy from userspace and the per-field checks, from
+ * the per-core tables in pvrsgx_cmd_defs.c. The per-command cross-field
+ * rules are in pvrsgx_cmd_checks.c.
+ */
+
+#include <linux/bits.h>
+#include <linux/errno.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+
+#include "pvrsgx_cmd_defs.h"
+
+static const struct pvrsgx_cmd_ext_def *ext_find(const struct pvrsgx_cmd_def *def, u32 id)
+{
+	for (u32 i = 0; i < def->nr_exts; i++) {
+		if (def->exts[i].id == id)
+			return &def->exts[i];
+	}
+	return NULL;
+}
+
+static int take_exts(const struct pvrsgx_cmd_def *def, u64 uptr, u8 *all, u64 *present)
+{
+	u32 n = 0;
+
+	*present = 0;
+	while (uptr) {
+		const struct pvrsgx_cmd_ext_def *ed;
+		struct drm_powervr_sgx_ext head;
+		u8 *dst;
+
+		if (++n > PVRSGX_CMD_MAX_EXTS)
+			return -EINVAL;
+		if (copy_from_user(&head, u64_to_user_ptr(uptr), sizeof(head)))
+			return -EFAULT;
+		if (head._padding_4 || !head.name || head.name >= 64 ||
+		    pvrsgx_cmd_ext_owner[head.name] != 1 + def->type)
+			return -EINVAL;	/* unknown, or another command's */
+		if (*present & BIT_ULL(head.name))
+			return -EINVAL;
+		ed = ext_find(def, head.name);
+		if (!ed)
+			return -EOPNOTSUPP;	/* this command's, but not on this core */
+		dst = all + ed->offset;
+		if (copy_from_user(dst, u64_to_user_ptr(uptr), ed->size))
+			return -EFAULT;
+		/* the copy is what counts: a racing writer can't change what was checked */
+		if (((struct drm_powervr_sgx_ext *)dst)->name != head.name)
+			return -EINVAL;
+		*present |= BIT_ULL(head.name);
+		uptr = ((struct drm_powervr_sgx_ext *)dst)->next;
+	}
+	return 0;
+}
+
+int pvrsgx_cmd_copy(const struct pvrsgx_cmd_def *def, u64 uptr, void *all, u64 *present)
+{
+	const void __user *src = u64_to_user_ptr(uptr);
+	struct drm_powervr_sgx_cmd_head *head = all;
+	u32 size;
+	int err;
+
+	memset(all, 0, def->all_size);
+	*present = 0;
+	if (!def->supported)
+		return -EOPNOTSUPP;
+	if (!uptr)
+		return -EINVAL;
+	if (get_user(size, (const u32 __user *)src))
+		return -EFAULT;
+	if (size < sizeof(*head) || size > PVRSGX_CMD_MAX_SIZE)
+		return -EINVAL;
+	/* -E2BIG for non-zero bytes past what this kernel knows, as everywhere */
+	err = copy_struct_from_user(all, def->size, src, size);
+	if (err)
+		return err;
+	if (head->size != size)
+		return -EINVAL;
+	return take_exts(def, head->extensions, all, present);
+}
+
+int pvrsgx_cmd_take_elem(const struct pvrsgx_cmd_def *def, const void *elem, u32 stride,
+			 void *all, u64 *present)
+{
+	const struct drm_powervr_sgx_cmd_head *head = elem;
+
+	memset(all, 0, def->all_size);
+	*present = 0;
+	if (!def->supported)
+		return -EOPNOTSUPP;
+	if (stride < sizeof(*head) || head->size != stride)
+		return -EINVAL;
+	/* obj_array_get() copied min(stride, def->size) and checked the tail */
+	memcpy(all, elem, def->size);
+	return take_exts(def, head->extensions, all, present);
+}
+
+static u64 cmd_field_get(const u8 *all, const struct pvrsgx_cmd_field *f)
+{
+	u64 v = 0;
+
+	memcpy(&v, all + f->offset, f->size);	/* little-endian: the low bytes */
+	return v;
+}
+
+static int check_field(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_field *f,
+		       const struct pvrsgx_cmd_ctx *ctx, u64 v)
+{
+	switch (f->check) {
+	case PVRSGX_CMD_CHECK_NONE:
+		return 0;
+	case PVRSGX_CMD_CHECK_ZERO:
+		return v ? -EINVAL : 0;
+	case PVRSGX_CMD_CHECK_RANGE: {
+		u64 max = f->limit ? ctx->limits[f->limit] : f->max;
+
+		return v < f->min || v > max ? -EINVAL : 0;
+	}
+	case PVRSGX_CMD_CHECK_MASK:
+		return v & ~f->mask ? -EINVAL : 0;
+	case PVRSGX_CMD_CHECK_ADDR:
+		if (!v)
+			return f->optional ? 0 : -EINVAL;
+		if (core->va_bits < 64 && v >> core->va_bits)
+			return -EINVAL;
+		if (v & (f->align - 1))
+			return -EINVAL;
+		if (f->heap == PVRSGX_CMD_HEAP_CLIENT)
+			return 0;	/* VM_MAP maps only client heaps; check_mapped covers it */
+		if (f->heap >= PVRSGX_CMD_HEAP_MAX || !ctx->heaps[f->heap].size ||
+		    v < ctx->heaps[f->heap].base ||
+		    v - ctx->heaps[f->heap].base >= ctx->heaps[f->heap].size)
+			return -EINVAL;
+		return 0;
+	}
+	return -EINVAL;
+}
+
+static int check_fields(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_field *fields,
+			u32 n, const struct pvrsgx_cmd_ctx *ctx, const u8 *all)
+{
+	for (u32 i = 0; i < n; i++) {
+		int err = check_field(core, &fields[i], ctx, cmd_field_get(all, &fields[i]));
+
+		if (err)
+			return err;
+	}
+	return 0;
+}
+
+int pvrsgx_cmd_check(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_def *def,
+		     const struct pvrsgx_cmd_ctx *ctx, const void *all, u64 present)
+{
+	const struct drm_powervr_sgx_cmd_head *head = all;
+	int err;
+
+	if (!def->supported)
+		return -EOPNOTSUPP;
+	if (head->_padding_4)
+		return -EINVAL;
+	err = check_fields(core, def->fields, def->nr_fields, ctx, all);
+	for (u32 i = 0; !err && i < def->nr_exts; i++) {
+		const struct pvrsgx_cmd_ext_def *ed = &def->exts[i];
+
+		if (present & BIT_ULL(ed->id))	/* absent: all zero, rev 1's "off" */
+			err = check_fields(core, ed->fields, ed->nr_fields, ctx, all);
+	}
+	return err;
+}
+
+static int mapped_fields(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_field *fields,
+			 u32 n, const struct pvrsgx_cmd_ctx *ctx, const u8 *all)
+{
+	for (u32 i = 0; i < n; i++) {
+		const struct pvrsgx_cmd_field *f = &fields[i];
+		u64 addr, size, samples = ctx->rt_samples ? ctx->rt_samples : 1;
+
+		if (f->check != PVRSGX_CMD_CHECK_ADDR || !f->extent)
+			continue;
+		addr = cmd_field_get(all, f);
+		if (!addr)
+			continue;	/* an optional address left out */
+		switch (f->extent) {
+		case PVRSGX_CMD_EXTENT_BYTES:
+			size = f->extent_bytes;
+			break;
+		case PVRSGX_CMD_EXTENT_RT_DEPTH:
+			size = (u64)ctx->rt_pad_width * ctx->rt_pad_height * 4 * samples;
+			break;
+		case PVRSGX_CMD_EXTENT_RT_STENCIL:
+			size = (u64)ctx->rt_pad_width * ctx->rt_pad_height * samples;
+			break;
+		case PVRSGX_CMD_EXTENT_PDS_ROWS: {
+			u32 rows;
+
+			memcpy(&rows, all + f->extent_field, sizeof(rows));
+			size = (u64)rows * core->pds_row_bytes;
+			break;
+		}
+		default:
+			return -EINVAL;
+		}
+		if (!size || !ctx->mapped(ctx->priv, addr, size))
+			return -EINVAL;
+	}
+	return 0;
+}
+
+int pvrsgx_cmd_check_mapped(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_def *def,
+			    const struct pvrsgx_cmd_ctx *ctx, const void *all, u64 present)
+{
+	int err = mapped_fields(core, def->fields, def->nr_fields, ctx, all);
+
+	for (u32 i = 0; !err && i < def->nr_exts; i++) {
+		const struct pvrsgx_cmd_ext_def *ed = &def->exts[i];
+
+		if (present & BIT_ULL(ed->id))
+			err = mapped_fields(core, ed->fields, ed->nr_fields, ctx, all);
+	}
+	return err;
+}
+
+u64 pvrsgx_cmd_ext_mask(const struct pvrsgx_cmd_def *def)
+{
+	u64 mask = 0;
+
+	for (u32 i = 0; def->supported && i < def->nr_exts; i++)
+		mask |= BIT_ULL(def->exts[i].id);
+	return mask;
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.h
new file mode 100644
index 000000000..ec5c29dff
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd.h
@@ -0,0 +1,132 @@
+/* SPDX-License-Identifier: MIT */
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Client commands: copied from userspace with their extensions, then checked
+ * field by field from the per-core tables in pvrsgx_cmd_defs.c.
+ */
+
+#ifndef PVRSGX_CMD_H
+#define PVRSGX_CMD_H
+
+#include <linux/types.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#define PVRSGX_CMD_MAX_SIZE 4096U
+#define PVRSGX_CMD_MAX_EXTS 16U
+#define PVRSGX_CMD_TYPES_MAX 8U
+
+enum pvrsgx_cmd_check {
+	PVRSGX_CMD_CHECK_NONE = 0,
+	PVRSGX_CMD_CHECK_ZERO,	/* padding */
+	PVRSGX_CMD_CHECK_RANGE,
+	PVRSGX_CMD_CHECK_MASK,
+	PVRSGX_CMD_CHECK_ADDR,
+};
+
+/* CLIENT has no range: VM_MAP only maps client heaps, and the mapped check covers it. */
+enum pvrsgx_cmd_heap {
+	PVRSGX_CMD_HEAP_CLIENT = 0,
+	PVRSGX_CMD_HEAP_PDS_PIXEL,
+	PVRSGX_CMD_HEAP_MAX,
+};
+
+enum pvrsgx_cmd_limit {
+	PVRSGX_CMD_LIMIT_NONE = 0,
+	PVRSGX_CMD_LIMIT_TILES_X,
+	PVRSGX_CMD_LIMIT_TILES_Y,
+	PVRSGX_CMD_LIMIT_MAX,
+};
+
+enum pvrsgx_cmd_extent {
+	PVRSGX_CMD_EXTENT_NONE = 0,
+	PVRSGX_CMD_EXTENT_BYTES,
+	PVRSGX_CMD_EXTENT_RT_DEPTH,
+	PVRSGX_CMD_EXTENT_RT_STENCIL,
+	PVRSGX_CMD_EXTENT_PDS_ROWS,
+};
+
+/* One field's check. Offsets are into the command's all-struct (pvrsgx_cmd_defs.h). */
+struct pvrsgx_cmd_field {
+	const char *name;
+	u16 offset;
+	u8 size;		/* 4 or 8 */
+	u8 check;
+	u8 limit;		/* RANGE: replaces max when set */
+	u8 heap;		/* ADDR */
+	u8 extent;		/* ADDR */
+	bool optional;		/* ADDR: 0 means none */
+	u16 extent_field;	/* PDS_ROWS: offset of the u32 row count */
+	u32 align;		/* ADDR */
+	u32 extent_bytes;	/* BYTES */
+	u64 min, max;		/* RANGE */
+	u64 mask;		/* MASK */
+};
+
+struct pvrsgx_cmd_ext_def {
+	u32 id;			/* DRM_POWERVR_SGX_EXT_* */
+	u16 size;		/* sizeof its struct */
+	u16 offset;		/* in the all-struct */
+	const struct pvrsgx_cmd_field *fields;
+	u32 nr_fields;
+};
+
+/* A command on one core. Not supported: a value it depends on is unknown there. */
+struct pvrsgx_cmd_def {
+	const char *name;
+	u8 type;		/* enum pvrsgx_cmd_type */
+	bool supported;
+	u16 size;		/* sizeof the base struct */
+	u16 all_size;		/* sizeof the all-struct: base and every extension */
+	const struct pvrsgx_cmd_field *fields;
+	u32 nr_fields;
+	const struct pvrsgx_cmd_ext_def *exts;	/* the extensions this core accepts */
+	u32 nr_exts;
+};
+
+struct pvrsgx_cmd_core {
+	const char *name;
+	u32 va_bits;
+	u32 pds_row_bytes;
+	u32 max_samples;	/* RT datasets: 1, or 4 where 4x is known to work */
+	const struct pvrsgx_cmd_def *cmds[PVRSGX_CMD_TYPES_MAX];
+};
+
+/* What a command is checked against: its VM, its RT, the device. */
+struct pvrsgx_cmd_ctx {
+	struct {
+		u64 base, size;
+	} heaps[PVRSGX_CMD_HEAP_MAX];	/* size 0: refuse every address */
+	u64 limits[PVRSGX_CMD_LIMIT_MAX];
+	u32 rt_pad_width, rt_pad_height;	/* the RT's size in whole tiles */
+	u32 rt_samples;				/* 0 is 1 */
+	bool (*mapped)(const void *priv, u64 addr, u64 size);
+	const void *priv;
+};
+
+/*
+ * A command at a user pointer, with its extensions, into @all (def->all_size
+ * bytes; absent extensions zero). @present gets a bit per extension sent.
+ */
+int pvrsgx_cmd_copy(const struct pvrsgx_cmd_def *def, u64 uptr, void *all, u64 *present);
+
+/*
+ * An array element obj_array_get() already copied at @stride (a TA kick):
+ * its head.size must equal @stride. Copies it into @all and walks its extensions.
+ */
+int pvrsgx_cmd_take_elem(const struct pvrsgx_cmd_def *def, const void *elem, u32 stride,
+			 void *all, u64 *present);
+
+/* Every field on its own: padding, ranges, masks, addresses' heaps and alignment. */
+int pvrsgx_cmd_check(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_def *def,
+		     const struct pvrsgx_cmd_ctx *ctx, const void *all, u64 present);
+
+/* Every address's extent is mapped in the VM (with the VM's lock held). */
+int pvrsgx_cmd_check_mapped(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_def *def,
+			    const struct pvrsgx_cmd_ctx *ctx, const void *all, u64 present);
+
+/* The extension names @def accepts, a bit per name (DEV_QUERY COMMANDS_GET). */
+u64 pvrsgx_cmd_ext_mask(const struct pvrsgx_cmd_def *def);
+
+#endif /* PVRSGX_CMD_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.c
new file mode 100644
index 000000000..ae0848f3c
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.c
@@ -0,0 +1,62 @@
+// SPDX-License-Identifier: MIT
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The cross-field rules of the client commands, run
+ * after pvrsgx_cmd_check() has checked each field on its own.
+ */
+
+#include <linux/bits.h>
+#include <linux/errno.h>
+
+#include "pvrsgx_cmd_checks.h"
+
+/* SGX545 ZLS control bits that use the stencil address. */
+#define ZLS_STENCIL_BITS 0x3U
+/* SGX545 3D flags bit that uses the stencil address. */
+#define FLAGS_3D_STENCIL BIT(8)
+/* ZLS addresses are offsets from a 1 MiB-aligned base; the 1 GiB limit is assumed. */
+#define ZLS_BASE_MASK 0xfff00000U
+#define ZLS_OFFSET_LIMIT_LOG2 30
+
+int pvrsgx_check_render(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_render_all *r,
+			u32 samples)
+{
+	/* The SGX545 extension; all zero when it wasn't sent or on another core. */
+	u32 zls = r->sgx545_render.zls_control, flags = r->sgx545_render.flags_3d;
+	u64 depth = r->base.depth_addr, stencil = r->base.stencil_addr, base, limit;
+
+	/* Multisampled depth and stencil are per sample and always stored. */
+	if (samples == 4 && !zls)
+		return -EINVAL;
+	if (zls && !depth)
+		return -EINVAL;
+	if (((zls & ZLS_STENCIL_BITS) || (flags & FLAGS_3D_STENCIL)) && !stencil)
+		return -EINVAL;
+
+	if (!depth && !stencil)
+		return 0;
+	if (!depth)
+		base = stencil;
+	else if (!stencil)
+		base = depth;
+	else
+		base = depth < stencil ? depth : stencil;
+	base &= ZLS_BASE_MASK;
+	limit = BIT_ULL(core->va_bits < ZLS_OFFSET_LIMIT_LOG2 ? core->va_bits :
+			ZLS_OFFSET_LIMIT_LOG2);
+
+	if (depth && depth - base >= limit)
+		return -EINVAL;
+	if (stencil && stencil - base >= limit)
+		return -EINVAL;
+	return 0;
+}
+
+int pvrsgx_check_transfer(const struct pvrsgx_cmd_transfer_all *t)
+{
+	if (t->base.first_tile_x > t->base.last_tile_x ||
+	    t->base.first_tile_y > t->base.last_tile_y)
+		return -EINVAL;
+	return 0;
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.h
new file mode 100644
index 000000000..06b14ae15
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_checks.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: MIT */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_CMD_CHECKS_H
+#define PVRSGX_CMD_CHECKS_H
+
+#include "pvrsgx_cmd_defs.h"
+
+int pvrsgx_check_render(const struct pvrsgx_cmd_core *core, const struct pvrsgx_cmd_render_all *r,
+			u32 samples);
+int pvrsgx_check_transfer(const struct pvrsgx_cmd_transfer_all *t);
+
+#endif /* PVRSGX_CMD_CHECKS_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.c
new file mode 100644
index 000000000..f73c80199
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.c
@@ -0,0 +1,353 @@
+// SPDX-License-Identifier: MIT
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The per-core client-command tables, generated by gen_cmd.py from the command
+ * description powervr_sgx_cmd.xml; both live in the driver's source repository,
+ * https://github.com/The1TrueJoe/powervr_sgx.
+ */
+
+#include <linux/array_size.h>
+#include <linux/build_bug.h>
+#include <linux/stddef.h>
+
+#include "pvrsgx_cmd_defs.h"
+
+/* The uAPI layout, pinned. */
+static_assert(PVRSGX_CMD_COUNT <= PVRSGX_CMD_TYPES_MAX);
+static_assert(sizeof(struct drm_powervr_sgx_cmd_render) == 80);
+static_assert(sizeof(struct drm_powervr_sgx_cmd_render) % 8 == 0);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, depth_addr) == 16);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, stencil_addr) == 24);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, pixel_event_pds_addr) == 32);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, bg_pds_secondary_addr) == 40);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, bg_pds_primary_addr) == 48);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, pixel_event_pds_rows) == 56);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, bg_pds_secondary_rows) == 60);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, bg_pds_primary_rows) == 64);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, bg_depth_f32) == 68);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, depth_bias_units) == 72);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_render, depth_bias_factor_f32) == 76);
+static_assert(sizeof(struct drm_powervr_sgx_cmd_ta_kick) == 24);
+static_assert(sizeof(struct drm_powervr_sgx_cmd_ta_kick) % 8 == 0);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_ta_kick, vdm_base) == 16);
+static_assert(sizeof(struct drm_powervr_sgx_cmd_transfer) == 56);
+static_assert(sizeof(struct drm_powervr_sgx_cmd_transfer) % 8 == 0);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_transfer, bg_block_addr) == 16);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_transfer, pixel_event_pds_addr) == 24);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_transfer, pixel_event_pds_rows) == 32);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_transfer, first_tile_x) == 36);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_transfer, first_tile_y) == 40);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_transfer, last_tile_x) == 44);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_transfer, last_tile_y) == 48);
+static_assert(offsetof(struct drm_powervr_sgx_cmd_transfer, bg_depth_f32) == 52);
+static_assert(sizeof(struct drm_powervr_sgx_ext_sgx545_render) == 24);
+static_assert(sizeof(struct drm_powervr_sgx_ext_sgx545_render) % 8 == 0);
+static_assert(offsetof(struct drm_powervr_sgx_ext_sgx545_render, zls_control) == 16);
+static_assert(offsetof(struct drm_powervr_sgx_ext_sgx545_render, flags_3d) == 20);
+static_assert(sizeof(struct drm_powervr_sgx_ext_sgx545_ta_kick) == 24);
+static_assert(sizeof(struct drm_powervr_sgx_ext_sgx545_ta_kick) % 8 == 0);
+static_assert(offsetof(struct drm_powervr_sgx_ext_sgx545_ta_kick, ta_flags2) == 16);
+
+const u8 pvrsgx_cmd_ext_owner[64] = {
+	[DRM_POWERVR_SGX_EXT_SGX545_RENDER] = 1 + PVRSGX_CMD_RENDER,
+	[DRM_POWERVR_SGX_EXT_SGX545_TA_KICK] = 1 + PVRSGX_CMD_TA_KICK,
+};
+
+static const struct pvrsgx_cmd_field sgx545_render_fields[] = {
+	{
+		.name = "depth_addr",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.depth_addr),
+		.size = 8,
+		.check = PVRSGX_CMD_CHECK_ADDR,
+		.heap = PVRSGX_CMD_HEAP_CLIENT,
+		.align = 1,
+		.optional = true,
+		.extent = PVRSGX_CMD_EXTENT_RT_DEPTH,
+	},
+	{
+		.name = "stencil_addr",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.stencil_addr),
+		.size = 8,
+		.check = PVRSGX_CMD_CHECK_ADDR,
+		.heap = PVRSGX_CMD_HEAP_CLIENT,
+		.align = 1,
+		.optional = true,
+		.extent = PVRSGX_CMD_EXTENT_RT_STENCIL,
+	},
+	{
+		.name = "pixel_event_pds_addr",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.pixel_event_pds_addr),
+		.size = 8,
+		.check = PVRSGX_CMD_CHECK_ADDR,
+		.heap = PVRSGX_CMD_HEAP_PDS_PIXEL,
+		.align = 16,
+		.extent = PVRSGX_CMD_EXTENT_PDS_ROWS,
+		.extent_field = offsetof(struct pvrsgx_cmd_render_all, base.pixel_event_pds_rows),
+	},
+	{
+		.name = "bg_pds_secondary_addr",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.bg_pds_secondary_addr),
+		.size = 8,
+		.check = PVRSGX_CMD_CHECK_ADDR,
+		.heap = PVRSGX_CMD_HEAP_PDS_PIXEL,
+		.align = 16,
+	},
+	{
+		.name = "bg_pds_primary_addr",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.bg_pds_primary_addr),
+		.size = 8,
+		.check = PVRSGX_CMD_CHECK_ADDR,
+		.heap = PVRSGX_CMD_HEAP_PDS_PIXEL,
+		.align = 16,
+	},
+	{
+		.name = "pixel_event_pds_rows",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.pixel_event_pds_rows),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_RANGE,
+		.min = 1,
+		.max = 63,
+	},
+	{
+		.name = "bg_pds_secondary_rows",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.bg_pds_secondary_rows),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_RANGE,
+		.min = 1,
+		.max = 63,
+	},
+	{
+		.name = "bg_pds_primary_rows",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.bg_pds_primary_rows),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_RANGE,
+		.min = 1,
+		.max = 63,
+	},
+	{
+		.name = "bg_depth_f32",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.bg_depth_f32),
+		.size = 4,
+	},
+	{
+		.name = "depth_bias_units",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.depth_bias_units),
+		.size = 4,
+	},
+	{
+		.name = "depth_bias_factor_f32",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, base.depth_bias_factor_f32),
+		.size = 4,
+	},
+};
+
+static const struct pvrsgx_cmd_field sgx545_sgx545_render_fields[] = {
+	{
+		.name = "zls_control",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, sgx545_render.zls_control),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_MASK,
+		.mask = 0x001ff007,
+	},
+	{
+		.name = "flags_3d",
+		.offset = offsetof(struct pvrsgx_cmd_render_all, sgx545_render.flags_3d),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_MASK,
+		.mask = 0x00000180,
+	},
+};
+
+static const struct pvrsgx_cmd_ext_def sgx545_render_exts[] = {
+	{
+		.id = DRM_POWERVR_SGX_EXT_SGX545_RENDER,
+		.size = sizeof(struct drm_powervr_sgx_ext_sgx545_render),
+		.offset = offsetof(struct pvrsgx_cmd_render_all, sgx545_render),
+		.fields = sgx545_sgx545_render_fields,
+		.nr_fields = ARRAY_SIZE(sgx545_sgx545_render_fields),
+	},
+};
+
+static const struct pvrsgx_cmd_def sgx545_render = {
+	.name = "render",
+	.type = PVRSGX_CMD_RENDER,
+	.supported = true,
+	.size = sizeof(struct drm_powervr_sgx_cmd_render),
+	.all_size = sizeof(struct pvrsgx_cmd_render_all),
+	.fields = sgx545_render_fields,
+	.nr_fields = ARRAY_SIZE(sgx545_render_fields),
+	.exts = sgx545_render_exts,
+	.nr_exts = ARRAY_SIZE(sgx545_render_exts),
+};
+
+static const struct pvrsgx_cmd_field sgx545_ta_kick_fields[] = {
+	{
+		.name = "vdm_base",
+		.offset = offsetof(struct pvrsgx_cmd_ta_kick_all, base.vdm_base),
+		.size = 8,
+		.check = PVRSGX_CMD_CHECK_ADDR,
+		.heap = PVRSGX_CMD_HEAP_CLIENT,
+		.align = 4,
+		.extent = PVRSGX_CMD_EXTENT_BYTES,
+		.extent_bytes = 4,
+	},
+};
+
+static const struct pvrsgx_cmd_field sgx545_sgx545_ta_kick_fields[] = {
+	{
+		.name = "ta_flags2",
+		.offset = offsetof(struct pvrsgx_cmd_ta_kick_all, sgx545_ta_kick.ta_flags2),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_MASK,
+		.mask = 0x02c80000,
+	},
+	{
+		.name = "_padding_14",
+		.offset = offsetof(struct pvrsgx_cmd_ta_kick_all, sgx545_ta_kick._padding_14),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_ZERO,
+	},
+};
+
+static const struct pvrsgx_cmd_ext_def sgx545_ta_kick_exts[] = {
+	{
+		.id = DRM_POWERVR_SGX_EXT_SGX545_TA_KICK,
+		.size = sizeof(struct drm_powervr_sgx_ext_sgx545_ta_kick),
+		.offset = offsetof(struct pvrsgx_cmd_ta_kick_all, sgx545_ta_kick),
+		.fields = sgx545_sgx545_ta_kick_fields,
+		.nr_fields = ARRAY_SIZE(sgx545_sgx545_ta_kick_fields),
+	},
+};
+
+static const struct pvrsgx_cmd_def sgx545_ta_kick = {
+	.name = "ta_kick",
+	.type = PVRSGX_CMD_TA_KICK,
+	.supported = true,
+	.size = sizeof(struct drm_powervr_sgx_cmd_ta_kick),
+	.all_size = sizeof(struct pvrsgx_cmd_ta_kick_all),
+	.fields = sgx545_ta_kick_fields,
+	.nr_fields = ARRAY_SIZE(sgx545_ta_kick_fields),
+	.exts = sgx545_ta_kick_exts,
+	.nr_exts = ARRAY_SIZE(sgx545_ta_kick_exts),
+};
+
+static const struct pvrsgx_cmd_field sgx545_transfer_fields[] = {
+	{
+		.name = "bg_block_addr",
+		.offset = offsetof(struct pvrsgx_cmd_transfer_all, base.bg_block_addr),
+		.size = 8,
+		.check = PVRSGX_CMD_CHECK_ADDR,
+		.heap = PVRSGX_CMD_HEAP_CLIENT,
+		.align = 16,
+		.extent = PVRSGX_CMD_EXTENT_BYTES,
+		.extent_bytes = 0x200,
+	},
+	{
+		.name = "pixel_event_pds_addr",
+		.offset = offsetof(struct pvrsgx_cmd_transfer_all, base.pixel_event_pds_addr),
+		.size = 8,
+		.check = PVRSGX_CMD_CHECK_ADDR,
+		.heap = PVRSGX_CMD_HEAP_PDS_PIXEL,
+		.align = 16,
+		.extent = PVRSGX_CMD_EXTENT_PDS_ROWS,
+		.extent_field = offsetof(struct pvrsgx_cmd_transfer_all, base.pixel_event_pds_rows),
+	},
+	{
+		.name = "pixel_event_pds_rows",
+		.offset = offsetof(struct pvrsgx_cmd_transfer_all, base.pixel_event_pds_rows),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_RANGE,
+		.min = 1,
+		.max = 63,
+	},
+	{
+		.name = "first_tile_x",
+		.offset = offsetof(struct pvrsgx_cmd_transfer_all, base.first_tile_x),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_RANGE,
+		.min = 0,
+		.limit = PVRSGX_CMD_LIMIT_TILES_X,
+	},
+	{
+		.name = "first_tile_y",
+		.offset = offsetof(struct pvrsgx_cmd_transfer_all, base.first_tile_y),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_RANGE,
+		.min = 0,
+		.limit = PVRSGX_CMD_LIMIT_TILES_Y,
+	},
+	{
+		.name = "last_tile_x",
+		.offset = offsetof(struct pvrsgx_cmd_transfer_all, base.last_tile_x),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_RANGE,
+		.min = 0,
+		.limit = PVRSGX_CMD_LIMIT_TILES_X,
+	},
+	{
+		.name = "last_tile_y",
+		.offset = offsetof(struct pvrsgx_cmd_transfer_all, base.last_tile_y),
+		.size = 4,
+		.check = PVRSGX_CMD_CHECK_RANGE,
+		.min = 0,
+		.limit = PVRSGX_CMD_LIMIT_TILES_Y,
+	},
+	{
+		.name = "bg_depth_f32",
+		.offset = offsetof(struct pvrsgx_cmd_transfer_all, base.bg_depth_f32),
+		.size = 4,
+	},
+};
+
+static const struct pvrsgx_cmd_def sgx545_transfer = {
+	.name = "transfer",
+	.type = PVRSGX_CMD_TRANSFER,
+	.supported = true,
+	.size = sizeof(struct drm_powervr_sgx_cmd_transfer),
+	.all_size = sizeof(struct pvrsgx_cmd_transfer_all),
+	.fields = sgx545_transfer_fields,
+	.nr_fields = ARRAY_SIZE(sgx545_transfer_fields),
+};
+
+const struct pvrsgx_cmd_core pvrsgx_cmd_core_sgx545 = {
+	.name = "sgx545",
+	.va_bits = 32,
+	.pds_row_bytes = 16,
+	.max_samples = 4,
+	.cmds = {
+		[PVRSGX_CMD_RENDER] = &sgx545_render,
+		[PVRSGX_CMD_TA_KICK] = &sgx545_ta_kick,
+		[PVRSGX_CMD_TRANSFER] = &sgx545_transfer,
+	},
+};
+
+/* sgx530: render depends on a value not yet known there. */
+static const struct pvrsgx_cmd_def sgx530_render = {
+	.name = "render",
+	.type = PVRSGX_CMD_RENDER,
+};
+
+/* sgx530: ta_kick depends on a value not yet known there. */
+static const struct pvrsgx_cmd_def sgx530_ta_kick = {
+	.name = "ta_kick",
+	.type = PVRSGX_CMD_TA_KICK,
+};
+
+/* sgx530: transfer depends on a value not yet known there. */
+static const struct pvrsgx_cmd_def sgx530_transfer = {
+	.name = "transfer",
+	.type = PVRSGX_CMD_TRANSFER,
+};
+
+const struct pvrsgx_cmd_core pvrsgx_cmd_core_sgx530 = {
+	.name = "sgx530",
+	.va_bits = 28,
+	.pds_row_bytes = 0,
+	.max_samples = 1,
+	.cmds = {
+		[PVRSGX_CMD_RENDER] = &sgx530_render,
+		[PVRSGX_CMD_TA_KICK] = &sgx530_ta_kick,
+		[PVRSGX_CMD_TRANSFER] = &sgx530_transfer,
+	},
+};
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.h
new file mode 100644
index 000000000..611e46e39
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_cmd_defs.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: MIT */
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The per-core client-command tables, generated by gen_cmd.py from the command
+ * description powervr_sgx_cmd.xml; both live in the driver's source repository,
+ * https://github.com/The1TrueJoe/powervr_sgx.
+ */
+
+#ifndef PVRSGX_CMD_DEFS_H
+#define PVRSGX_CMD_DEFS_H
+
+#include "pvrsgx_cmd.h"
+
+enum pvrsgx_cmd_type {
+	PVRSGX_CMD_RENDER,
+	PVRSGX_CMD_TA_KICK,
+	PVRSGX_CMD_TRANSFER,
+	PVRSGX_CMD_COUNT,
+};
+
+/* A command with every extension it can take on any core: absent ones zero. */
+struct pvrsgx_cmd_render_all {
+	struct drm_powervr_sgx_cmd_render base;
+	struct drm_powervr_sgx_ext_sgx545_render sgx545_render;
+};
+
+struct pvrsgx_cmd_ta_kick_all {
+	struct drm_powervr_sgx_cmd_ta_kick base;
+	struct drm_powervr_sgx_ext_sgx545_ta_kick sgx545_ta_kick;
+};
+
+struct pvrsgx_cmd_transfer_all {
+	struct drm_powervr_sgx_cmd_transfer base;
+};
+
+/* Extension name -> 1 + the command it belongs to, 0 for none. */
+extern const u8 pvrsgx_cmd_ext_owner[64];
+
+extern const struct pvrsgx_cmd_core pvrsgx_cmd_core_sgx545;
+extern const struct pvrsgx_cmd_core pvrsgx_cmd_core_sgx530;
+
+#endif /* PVRSGX_CMD_DEFS_H */

-- 
2.50.1

[PATCH RFC 10/13] drm/powervr_sgx: Add render jobs

[Joseph Telaak]

- A render job is the client's TA kicks and render, built by the kernel
  into the microkernel's TA command from checked command fields.
- One parameter buffer (the tiler's output) serves every client, mapped
  outside every client heap; its first render context sets its size.
- Render target datasets: render details, region headers and tail
  pointers, kernel-owned.
- Transfer jobs (3D-only blits), which only the test tools use so far.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/powervr_sgx/pvrsgx_render.c   | 952 ++++++++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_render.h   | 122 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.c | 175 +++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.h |  38 +
 4 files changed, 1287 insertions(+)

diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_render.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_render.c
new file mode 100644
index 000000000..00b444d05
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_render.c
@@ -0,0 +1,952 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Render contexts, RT datasets and render jobs: everything the microkernel
+ * needs for a TA kick and the 3D after it, built by the kernel from the
+ * client's CMD_RENDER and CMD_TA_KICK streams.
+ *
+ * A render job is one scene of one TA kick, which starts the scene and the 3D
+ * (TA flags 0x63). Scenes of several kicks need the middle-kick flags, which
+ * are not known yet.
+ */
+
+#include <drm/drm_file.h>
+#include <drm/drm_gpuvm.h>
+#include <drm/drm_print.h>
+
+#include <linux/delay.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/seq_buf.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_cmd.h"
+#include "pvrsgx_cmd_checks.h"
+#include "pvrsgx_ccb.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_drv.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_fwif.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_kobj.h"
+#include "pvrsgx_mmu.h"
+#include "pvrsgx_pm.h"
+#include "pvrsgx_render.h"
+#include "pvrsgx_trace.h"
+#include "pvrsgx_transfer.h"
+#include "pvrsgx_vm.h"
+
+#define PB_EXTRA_PAGES		0x22
+#define FREELIST_BYTES		0x40000	/* one word per page of the 256 MiB range */
+#define CCB_BYTES		0x10420
+/* The background object's middle PDS word, a constant. */
+#define BG_PDS_MIDDLE		0x00030001
+/* Constant TA command words. */
+#define TA_CTL			0x50000000
+#define TE_AA			0x20000000
+#define TE_AA_4X		(TE_AA | BIT(31) | BIT(30))	/* 2x in X and in Y */
+#define VDM_CTL			0xa2
+#define F32_1E_6		0x358637bd
+#define RT_FLAGS		0x88
+#define RT_FLAGS_4X		0xeaa26e26	/* the RT flags of a 4x render */
+#define RT_MAX_DIM		2048
+
+static u32 f32_one(void)
+{
+	return 0x3f800000;
+}
+
+/* ---------------------------------------------------------------- render context */
+
+/* The PB's free list and descriptor as new: at creation, and after a reset (pb_rearm). */
+static void pb_init(struct pvrsgx_pb *pb)
+{
+	u32 pages = pb->pages, first, last, zls, ta, pds, *fl;
+	struct pvrsgx_fw_pb_desc *pd;
+
+	/* Free list: prev<<16 | next per page, indexed from the range's base. */
+	first = (kobj_va(pb->buf) - pvrsgx_heaps(pb->sdev)->params_3d.base) >> PAGE_SHIFT;
+	last = first + pages - 1;
+	fl = kobj_cpu(pb->freelist);
+	for (u32 i = first; i <= last; i++)
+		fl[i] = (i == first ? 0xffffu : i - 1) << 16 | (i == last ? 0xffffu : i + 1);
+
+	kobj_cpu(pb->blocks)[0] = pages;
+	kobj_cpu(pb->blocks)[1] = first | last << 16;
+	kobj_cpu(pb->blocks)[2] = kobj_va(pb->desc);
+
+	zls = pages - PB_EXTRA_PAGES;
+	ta = pages + 0x10;
+	pds = min(zls, ta);
+	pd = (struct pvrsgx_fw_pb_desc *)kobj_cpu(pb->desc);
+	memset(pd, 0, sizeof(*pd));
+	pd->pages = pages;
+	pd->free_head = last << 16 | first;
+	pd->free_tail = (last - 1) << 16;
+	pd->free_list = kobj_va(pb->freelist);
+	pd->free_head_copy = pd->free_head;
+	pd->free_tail_copy = pd->free_tail;
+	pd->ta_threshold = ta;
+	pd->zls_threshold = zls;
+	pd->global_threshold = zls;
+	pd->pds_threshold = pds > 0x100 ? pds - 0x100 : 0;
+	pd->blocks = kobj_va(pb->blocks);
+}
+
+/*
+ * The device's parameter buffer: PB, free list, PB descriptor and block list,
+ * shared by every render context of every client. The DPM's free-list state is
+ * one set of registers that the microkernel doesn't switch between PBs, so two
+ * PBs would run each other's lists off their ends. The DPM numbers PB pages from
+ * the 3D-params heap base, so the PB starts there, at the same address in every
+ * VM (pvrsgx_pb_map). The first context sets the size; later requests for
+ * another size are refused.
+ */
+static struct pvrsgx_pb *pb_create(struct pvrsgx_device *sdev, u32 req_pages)
+{
+	struct pvrsgx_pb *pb = kzalloc_obj(*pb);
+	u32 pages = (req_pages ?: PVRSGX_PB_DEFAULT_PAGES) + PB_EXTRA_PAGES;
+	int err;
+
+	if (!pb)
+		return ERR_PTR(-ENOMEM);
+	kref_init(&pb->ref);
+	pb->sdev = sdev;
+
+	/* The PB, and what is addressed from its 1 MiB-aligned base, in the VM. */
+	err = pvrsgx_kobj_new(sdev, &pb->buf, PVRSGX_KRANGE_3DPARAMS, pages << PAGE_SHIFT, 0, NULL);
+	if (err)
+		goto fail;
+	pb->base = kobj_va(pb->buf) & 0xfff00000;
+	err = pvrsgx_kobj_new(sdev, &pb->freelist, PVRSGX_KRANGE_3DPARAMS, FREELIST_BYTES, pb->base,
+			      NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &pb->desc, PVRSGX_KRANGE_KDATA,
+				     sizeof(struct pvrsgx_fw_pb_desc), 0, NULL);
+	if (pb->desc)
+		pb->desc->until_restart = true;
+	err = err ?: pvrsgx_kobj_new(sdev, &pb->blocks, PVRSGX_KRANGE_KDATA, 0x10, 0, NULL);
+	if (err)
+		goto fail;
+
+	pb->pages = pages;
+	pb_init(pb);
+	pb->fw_gen = sdev->fw_gen;
+	return pb;
+
+fail:
+	pvrsgx_pb_put(pb);
+	return ERR_PTR(err);
+}
+
+static void pb_release(struct kref *ref)
+{
+	struct pvrsgx_pb *pb = container_of(ref, struct pvrsgx_pb, ref);
+	struct pvrsgx_device *sdev = pb->sdev;
+
+	pvrsgx_kobj_drop(sdev, &pb->blocks);
+	pvrsgx_kobj_drop(sdev, &pb->desc);
+	pvrsgx_kobj_drop(sdev, &pb->freelist);
+	pvrsgx_kobj_drop(sdev, &pb->buf);
+	kfree(pb);
+}
+
+void pvrsgx_pb_put(struct pvrsgx_pb *pb)
+{
+	if (pb)
+		kref_put(&pb->ref, pb_release);
+}
+
+/* The device's PB, created by the first render context; the device keeps one reference. */
+static struct pvrsgx_pb *dev_pb_get(struct pvrsgx_device *sdev, u32 req_pages)
+{
+	struct pvrsgx_pb *pb;
+
+	mutex_lock(&sdev->pb_lock);
+	if (!sdev->pb) {
+		pb = pb_create(sdev, req_pages);
+		if (IS_ERR(pb)) {
+			mutex_unlock(&sdev->pb_lock);
+			return pb;
+		}
+		sdev->pb = pb;
+		drm_info(&sdev->base, "parameter buffer: %#x pages\n", pb->pages);
+	} else if (req_pages && req_pages != sdev->pb->pages - PB_EXTRA_PAGES) {
+		/* A size the live buffer doesn't have is refused, not ignored. */
+		mutex_unlock(&sdev->pb_lock);
+		return ERR_PTR(-EINVAL);
+	}
+	pb = sdev->pb;
+	kref_get(&pb->ref);
+	mutex_unlock(&sdev->pb_lock);
+	return pb;
+}
+
+static struct pvrsgx_kobj *pb_mapped(struct pvrsgx_pb *pb, int i)
+{
+	return i ? pb->freelist : pb->buf;
+}
+
+/* The PB and its free list into the VM, at their device addresses; once per VM. */
+static int pvrsgx_pb_map(struct pvrsgx_vm *vm, struct pvrsgx_pb *pb)
+{
+	struct pvrsgx_device *sdev = vm->sdev;
+	int err = 0, i;
+
+	mutex_lock(&vm->lock);
+	if (vm->pb)
+		goto out;
+	for (i = 0; i < ARRAY_SIZE(vm->pb_nodes); i++) {
+		struct pvrsgx_kobj *o = pb_mapped(pb, i);
+
+		vm->pb_nodes[i].start = kobj_va(o);
+		vm->pb_nodes[i].size = o->km.size;
+		mutex_lock(&sdev->kobj_lock);
+		err = drm_mm_reserve_node(&vm->params, &vm->pb_nodes[i]);
+		mutex_unlock(&sdev->kobj_lock);
+		if (err)
+			goto unwind;
+		err = pvrsgx_mmu_map_pages(vm->mmu, kobj_va(o), o->km.pages, o->km.npages, 0);
+		if (err) {
+			i++;
+			goto unwind;
+		}
+	}
+	kref_get(&pb->ref);
+	vm->pb = pb;
+	mutex_lock(&sdev->ccb_lock);
+	sdev->fw.cache_control |= PVRSGX_CC_INVAL_BIF_PT | PVRSGX_CC_INVAL_BIF_PD;
+	mutex_unlock(&sdev->ccb_lock);
+out:
+	mutex_unlock(&vm->lock);
+	return err;
+
+unwind:
+	while (i--) {
+		pvrsgx_mmu_unmap(vm->mmu, vm->pb_nodes[i].start, vm->pb_nodes[i].size);
+		mutex_lock(&sdev->kobj_lock);
+		drm_mm_remove_node(&vm->pb_nodes[i]);
+		mutex_unlock(&sdev->kobj_lock);
+	}
+	mutex_unlock(&vm->lock);
+	return err;
+}
+
+/* At the VM's release: nothing on the GPU uses it any more. */
+void pvrsgx_pb_unmap(struct pvrsgx_vm *vm)
+{
+	struct pvrsgx_device *sdev = vm->sdev;
+
+	for (int i = 0; i < ARRAY_SIZE(vm->pb_nodes); i++) {
+		pvrsgx_mmu_unmap(vm->mmu, vm->pb_nodes[i].start, vm->pb_nodes[i].size);
+		mutex_lock(&sdev->kobj_lock);
+		drm_mm_remove_node(&vm->pb_nodes[i]);
+		mutex_unlock(&sdev->kobj_lock);
+	}
+	pvrsgx_pb_put(vm->pb);
+	vm->pb = NULL;
+}
+
+int pvrsgx_render_ctx_init(struct pvrsgx_device *sdev, struct pvrsgx_context *ctx, u32 req_pages)
+{
+	struct pvrsgx_render_ctx *r = kzalloc_obj(*r);
+	struct pvrsgx_vm *vm = ctx->vm;
+	struct pvrsgx_fw_hw_render_ctx *hw;
+	int err = -ENOMEM;
+
+	if (!r)
+		return -ENOMEM;
+	ctx->render = r;
+
+	r->pb = dev_pb_get(sdev, req_pages);
+	if (IS_ERR(r->pb)) {
+		err = PTR_ERR(r->pb);
+		r->pb = NULL;
+		goto fail;
+	}
+	err = pvrsgx_pb_map(vm, r->pb);
+	if (err)
+		goto fail;
+
+	err = pvrsgx_kobj_new(sdev, &r->scratch, PVRSGX_KRANGE_3DPARAMS, 0x40, r->pb->base, vm);
+	err = err ?: pvrsgx_kobj_new(sdev, &r->ccb, PVRSGX_KRANGE_KDATA, CCB_BYTES, 0, NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &r->ccbctl, PVRSGX_KRANGE_KDATA, 8, 0, NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &r->sync3d, PVRSGX_KRANGE_SYNC, SYNC_DATA_BYTES, 0,
+				     NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &r->hwctx, PVRSGX_KRANGE_KCODE,
+				     sizeof(struct pvrsgx_fw_hw_render_ctx), 0, NULL);
+	if (err)
+		goto fail;
+
+	/* HW render context: the microkernel points list 0 at word 2's directory. */
+	hw = (void *)kobj_cpu(r->hwctx);
+	hw->flags = PVRSGX_HW_RENDER_CTX_VALID | PVRSGX_HW_RENDER_CTX_OWN_PB;
+	hw->page_dir = pvrsgx_vm_pd_word(vm);
+	hw->ccb = kobj_va(r->ccb);
+	hw->ccb_ctl = kobj_va(r->ccbctl);
+	hw->pb_desc = kobj_va(r->pb->desc);
+	hw->scratch = kobj_va(r->scratch);
+	hw->pid = task_tgid_nr(current);
+	return 0;
+
+fail:
+	pvrsgx_render_ctx_fini(sdev, ctx);
+	return err;
+}
+
+void pvrsgx_render_ctx_fini(struct pvrsgx_device *sdev, struct pvrsgx_context *ctx)
+{
+	struct pvrsgx_render_ctx *r = ctx->render;
+
+	if (!r)
+		return;
+	pvrsgx_kobj_drop(sdev, &r->hwctx);
+	pvrsgx_kobj_drop(sdev, &r->sync3d);
+	pvrsgx_kobj_drop(sdev, &r->ccbctl);
+	pvrsgx_kobj_drop(sdev, &r->ccb);
+	pvrsgx_kobj_drop(sdev, &r->scratch);
+	pvrsgx_pb_put(r->pb);
+	kfree(r);
+	ctx->render = NULL;
+}
+
+/* ---------------------------------------------------------------- RT datasets */
+
+/*
+ * At 4x (2x2 samples) the TE tiles samples: twice the tiles in X and in Y,
+ * and the region and tail pointer arrays follow. Its macrotile registers
+ * stay in pixel tiles, as does the screen.
+ */
+static void rt_numbers(struct pvrsgx_rt_numbers *r, u32 w, u32 h, u32 samples)
+{
+	u32 aa = samples == 4 ? 2 : 1, xt, yt, mx, my, n, big, m;
+
+	r->xt = (w + 31) >> 5;
+	r->yt = (h + 15) >> 4;
+	xt = r->xt * aa;
+	yt = r->yt * aa;
+	if (w * h * samples < 0x4b000) {
+		mx = (((xt + 1) >> 1) + 3) & ~3u;
+		my = (((yt + 1) >> 1) + 3) & ~3u;
+		n = 2;
+	} else {
+		mx = (((xt + 3) >> 2) + 3) & ~3u;
+		my = (((yt + 3) >> 2) + 3) & ~3u;
+		n = 4;
+	}
+	r->mx = mx / aa;
+	r->my = my / aa;
+	if (n == 2) {
+		r->mtile1 = r->mx << 22 | r->mx << 12 | r->mx;
+		r->mtile2 = r->my << 22 | r->my << 12 | r->my;
+	} else {
+		r->mtile1 = BIT(31) | r->mx << 22 | 2 * r->mx << 12 | 3 * r->mx;
+		r->mtile2 = r->my << 22 | 2 * r->my << 12 | 3 * r->my;
+	}
+	r->zls_xt = xt;
+	r->padx = mx * n;
+	r->pady = my * n;
+	r->te_screen = (r->yt - 1) << 12 | (r->xt - 1);
+	r->mt_tiles = r->mx * r->my;
+	r->rt_mt_tiles = mx * my;
+	r->region_bytes = r->padx * r->pady * 12;
+	m = max(r->padx, r->pady) - 1;
+	for (big = 1; m; m >>= 1)
+		big <<= 1;
+	r->tailptr_bytes = big * big * 4;
+	r->te_aa = aa == 2 ? TE_AA_4X : TE_AA;
+	r->rt_flags = aa == 2 ? RT_FLAGS_4X : RT_FLAGS;
+}
+
+static void rtds_release(struct kref *ref)
+{
+	struct pvrsgx_rt_dataset *s = container_of(ref, struct pvrsgx_rt_dataset, ref);
+	struct pvrsgx_device *sdev = s->ctx->sdev;
+
+	pvrsgx_kobj_drop(sdev, &s->rtword);
+	pvrsgx_kobj_drop(sdev, &s->rtdata);
+	pvrsgx_kobj_drop(sdev, &s->done);
+	pvrsgx_kobj_drop(sdev, &s->tailptr);
+	pvrsgx_kobj_drop(sdev, &s->ta40);
+	pvrsgx_kobj_drop(sdev, &s->ta50);
+	for (int i = 0; i < 2; i++) {
+		pvrsgx_kobj_drop(sdev, &s->bgblock[i]);
+		pvrsgx_kobj_drop(sdev, &s->rd[i]);
+		pvrsgx_kobj_drop(sdev, &s->rd_w8[i]);
+		pvrsgx_kobj_drop(sdev, &s->rd_w4[i]);
+		pvrsgx_kobj_drop(sdev, &s->hwsync[i]);
+		pvrsgx_kobj_drop(sdev, &s->rd_sync[i]);
+		pvrsgx_kobj_drop(sdev, &s->hwsync_access[i]);
+		pvrsgx_kobj_drop(sdev, &s->entword[i]);
+		pvrsgx_kobj_drop(sdev, &s->rgn[i]);
+		pvrsgx_kobj_drop(sdev, &s->ta88[i]);
+	}
+	pvrsgx_context_put(s->ctx);
+	kfree(s);
+}
+
+void pvrsgx_rtds_put(struct pvrsgx_rt_dataset *rtds)
+{
+	if (rtds)
+		kref_put(&rtds->ref, rtds_release);
+}
+
+/* The dataset's structures as new, frame count 0. */
+/* One RT-data entry's render details and dependency block, as new. */
+static void rtds_init_entry(struct pvrsgx_rt_dataset *s, int i)
+{
+	struct pvrsgx_render_ctx *r = s->ctx->render;
+	struct pvrsgx_fw_render_details *rd = (void *)kobj_cpu(s->rd[i]);
+
+	memset(rd, 0, sizeof(*rd));
+	/* Render details; the per-render words are written at each job. */
+	rd->req_base_3d = r->pb->base;
+	rd->unk_08 = 0x80;
+	rd->unk_0c = 0x000f0000;
+	rd->rt_ctl = BIT(31) | (s->samples == 4 ? BIT(1) : 0);
+	rd->stencil[0] = BIT(31);
+	rd->stencil[1] = BIT(31);
+	rd->bg_depth = f32_one();
+	rd->unk_44 = 0x300;
+	rd->bg_object = 0x2000000 | (kobj_va(s->bgblock[i]) - r->pb->base) >> 4;
+	rd->rt_flags = s->n.rt_flags;
+	rd->unk_58 = 0x30;
+	rd->unk_64 = 1;
+	rd->unk_68 = 1;
+	rd->unk_cc = 0x100;
+	rd->sync = kobj_va(s->rd_sync[i]);
+	rd->rt_data = kobj_va(s->rtdata);
+	rd->unk_104 = 1;
+	rd->word8 = kobj_va(s->rd_w8[i]);
+	rd->bg_pds_word = kobj_va(s->rd_w4[i]);
+	rd->unk_13c = 0x07c00000;
+	kobj_cpu(s->rd_sync[i])[0] = 1;
+	kobj_cpu(s->hwsync[i])[0] = kobj_va(s->hwsync_access[i]);
+	kobj_cpu(s->hwsync_access[i])[0] = 1;
+}
+
+static void rtds_init(struct pvrsgx_rt_dataset *s)
+{
+	struct pvrsgx_render_ctx *r = s->ctx->render;
+	struct pvrsgx_fw_rt_data *rt = (void *)kobj_cpu(s->rtdata);
+	u32 w = s->w, h = s->h, *b;
+
+	memset(rt, 0, sizeof(*rt));
+	for (int i = 0; i < 2; i++) {
+		memset(kobj_cpu(s->bgblock[i]), 0, 0x60);
+		kobj_cpu(s->entword[i])[0] = 0;
+	}
+	s->frames = 0;
+
+	kobj_cpu(s->rtword)[0] = 1;
+	rt->rt_word = kobj_va(s->rtword);
+	rt->padded_tiles_x = s->n.padx;
+	rt->num_entries = ARRAY_SIZE(rt->entry);
+	for (int i = 0; i < ARRAY_SIZE(rt->entry); i++) {
+		struct pvrsgx_fw_rt_entry *e = &rt->entry[i];
+
+		e->hw_render_ctx = kobj_va(r->hwctx);
+		e->mt_units = s->n.rt_mt_tiles >> 4;
+		e->mt_bytes = s->n.rt_mt_tiles * 0xc;
+		e->bg_block = kobj_va(s->bgblock[i]);
+		e->ta_data[0] = kobj_va(s->ta88[i]);
+		e->ta_data[1] = kobj_va(s->ta40);
+		e->ta_data[2] = kobj_va(s->ta50);
+		e->tail_pointers = kobj_va(s->tailptr);
+		e->tail_pointers_size = s->n.tailptr_bytes;
+		e->num_rts = 1;
+		e->entry_word = kobj_va(s->entword[i]);
+		e->unk_34 = 0x10;
+	}
+
+	for (int i = 0; i < 2; i++) {
+		/* Background object: screen-space vertices, fixed point offset by 0x1000. */
+		b = kobj_cpu(s->bgblock[i]);
+		b[5] = f32_one();
+		b[8] = f32_one();
+		b[10] = 0x202;
+		b[15] = 0x100000;
+		b[16] = 0x1000;
+		b[17] = 0x3f80;
+		b[18] = ((w + 0x1000) * 0x100) << 8 | 0x10;
+		b[19] = f32_one();
+		b[20] = (h + 0x1000) * 0x100;
+		b[21] = 0x1000;
+		b[22] = 0x3f80;
+		rtds_init_entry(s, i);
+	}
+}
+
+struct pvrsgx_rt_dataset *pvrsgx_rtds_create(struct pvrsgx_context *ctx, u32 w, u32 h, u32 samples)
+{
+	struct pvrsgx_device *sdev = ctx->sdev;
+	struct pvrsgx_render_ctx *r = ctx->render;
+	struct pvrsgx_vm *vm = ctx->vm;
+	struct pvrsgx_rt_dataset *s = kzalloc_obj(*s);
+	int err;
+
+	if (!s)
+		return ERR_PTR(-ENOMEM);
+	kref_init(&s->ref);
+	kref_get(&ctx->ref);
+	s->ctx = ctx;
+	s->w = w;
+	s->h = h;
+	s->samples = samples;
+	rt_numbers(&s->n, w, h, samples);
+
+	err = pvrsgx_kobj_new(sdev, &s->rtword, PVRSGX_KRANGE_KDATA, 4, 0, NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &s->rtdata, PVRSGX_KRANGE_KDATA,
+				     sizeof(struct pvrsgx_fw_rt_data), 0, NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &s->done, PVRSGX_KRANGE_SYNC, SYNC_DATA_BYTES, 0, NULL);
+	/* What the TA writes and the 3D reads, under the VM's directory too. */
+	err = err ?: pvrsgx_kobj_new(sdev, &s->tailptr, PVRSGX_KRANGE_TADATA,
+				     s->n.tailptr_bytes, 0, vm);
+	err = err ?: pvrsgx_kobj_new(sdev, &s->ta40, PVRSGX_KRANGE_TADATA, 0x40, 0, vm);
+	err = err ?: pvrsgx_kobj_new(sdev, &s->ta50, PVRSGX_KRANGE_TADATA, 0x50, 0, vm);
+	for (int i = 0; i < 2; i++) {
+		err = err ?: pvrsgx_kobj_new(sdev, &s->bgblock[i], PVRSGX_KRANGE_3DPARAMS, 0x200,
+					     r->pb->base, vm);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->rgn[i], PVRSGX_KRANGE_TADATA,
+					     s->n.region_bytes, 0, vm);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->ta88[i], PVRSGX_KRANGE_TADATA, 0x88, 0, vm);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->entword[i], PVRSGX_KRANGE_KDATA, 4, 0, NULL);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->rd[i], PVRSGX_KRANGE_KDATA,
+					     sizeof(struct pvrsgx_fw_render_details), 0, NULL);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->rd_w8[i], PVRSGX_KRANGE_KDATA, 8, 0, NULL);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->rd_w4[i], PVRSGX_KRANGE_KDATA, 4, 0, NULL);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->hwsync[i], PVRSGX_KRANGE_KDATA,
+					     sizeof(struct pvrsgx_fw_rt_deps), 0, NULL);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->rd_sync[i], PVRSGX_KRANGE_SYNC, 8, 0, NULL);
+		err = err ?: pvrsgx_kobj_new(sdev, &s->hwsync_access[i], PVRSGX_KRANGE_SYNC, 8, 0,
+					     NULL);
+	}
+	if (err)
+		goto fail;
+
+	rtds_init(s);
+	return s;
+
+fail:
+	pvrsgx_rtds_put(s);
+	return ERR_PTR(err);
+}
+
+static struct pvrsgx_rt_dataset *rtds_lookup(struct pvrsgx_file *sfile, u32 handle)
+{
+	struct pvrsgx_rt_dataset *s;
+
+	xa_lock(&sfile->rtds);
+	s = xa_load(&sfile->rtds, handle);
+	if (s)
+		kref_get(&s->ref);
+	xa_unlock(&sfile->rtds);
+	return s;
+}
+
+void pvrsgx_rtds_destroy_all(struct pvrsgx_file *sfile)
+{
+	struct pvrsgx_rt_dataset *s;
+	unsigned long handle;
+
+	xa_for_each(&sfile->rtds, handle, s) {
+		xa_erase(&sfile->rtds, handle);
+		pvrsgx_rtds_put(s);
+	}
+}
+
+int pvrsgx_ioctl_create_rt_dataset(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_create_rt_dataset_args *args = data;
+	struct pvrsgx_file *sfile = to_pvrsgx_file(file);
+	struct pvrsgx_rt_dataset *s;
+	struct pvrsgx_context *ctx;
+	int err;
+
+	if (args->flags || (args->samples > 1 && args->samples != 4) || !args->width ||
+	    !args->height ||
+	    args->width > RT_MAX_DIM || args->height > RT_MAX_DIM)
+		return -EINVAL;
+	if (args->samples > to_pvrsgx_device(drm)->family->cmds->max_samples)
+		return -EOPNOTSUPP;
+	ctx = pvrsgx_context_lookup(sfile, args->context_handle);
+	if (!ctx)
+		return -ENOENT;
+	if (!ctx->render) {
+		pvrsgx_context_put(ctx);
+		return -EINVAL;
+	}
+	s = pvrsgx_rtds_create(ctx, args->width, args->height, args->samples ?: 1);
+	pvrsgx_context_put(ctx);
+	if (IS_ERR(s))
+		return PTR_ERR(s);
+	err = xa_alloc(&sfile->rtds, &args->handle, s, XA_LIMIT(1, PVRSGX_MAX_FILE_RTDS),
+		       GFP_KERNEL);
+	if (err) {
+		pvrsgx_rtds_put(s);
+		return err == -EBUSY ? -ENOSPC : err;
+	}
+	return 0;
+}
+
+int pvrsgx_ioctl_destroy_rt_dataset(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_destroy_rt_dataset_args *args = data;
+	struct pvrsgx_rt_dataset *s;
+
+	if (args->_padding_4)
+		return -EINVAL;
+	s = xa_erase(&to_pvrsgx_file(file)->rtds, args->handle);
+	if (!s)
+		return -ENOENT;
+	pvrsgx_rtds_put(s);
+	return 0;
+}
+
+/* ---------------------------------------------------------------- streams */
+
+bool pvrsgx_cmd_mapped(const void *priv, u64 addr, u64 size)
+{
+	struct pvrsgx_vm *vm = (struct pvrsgx_vm *)priv;
+	u64 cur = addr, end = addr + size;
+	struct drm_gpuva *va;
+
+	if (end < addr)
+		return false;
+	drm_gpuvm_for_each_va_range(va, &vm->gpuvm, addr, end) {
+		if (va->va.addr > cur)
+			return false;
+		cur = va->va.addr + va->va.range;
+		if (cur >= end)
+			return true;
+	}
+	return cur >= end;
+}
+
+/* The core's command of @type: what a client's command is checked against. */
+const struct pvrsgx_cmd_def *pvrsgx_cmd_def(struct pvrsgx_device *sdev, enum pvrsgx_cmd_type type)
+{
+	return sdev->family->cmds->cmds[type];
+}
+
+/* A copied command, field by field, then against the VM (under its lock). */
+int pvrsgx_cmd_take(struct pvrsgx_device *sdev, const struct pvrsgx_cmd_def *def,
+		    const struct pvrsgx_cmd_ctx *ctx, struct pvrsgx_vm *vm, const void *all,
+		    u64 present)
+{
+	const struct pvrsgx_cmd_core *core = sdev->family->cmds;
+	int err = pvrsgx_cmd_check(core, def, ctx, all, present);
+
+	if (err)
+		return err;
+	mutex_lock(&vm->lock);
+	err = pvrsgx_cmd_check_mapped(core, def, ctx, all, present);
+	mutex_unlock(&vm->lock);
+	return err;
+}
+
+int pvrsgx_render_job_init(struct drm_file *file, struct pvrsgx_job *job,
+			   const struct drm_powervr_sgx_job *in)
+{
+	struct pvrsgx_device *sdev = job->ctx->sdev;
+	const struct pvrsgx_cmd_def *rdef = pvrsgx_cmd_def(sdev, PVRSGX_CMD_RENDER);
+	const struct pvrsgx_cmd_def *kdef = pvrsgx_cmd_def(sdev, PVRSGX_CMD_TA_KICK);
+	struct pvrsgx_render_job *rj = &job->render;
+	struct drm_powervr_sgx_cmd_ta_kick *kicks;
+	struct pvrsgx_vm *vm = job->ctx->vm;
+	struct pvrsgx_cmd_ctx cctx = {};
+	u64 present;
+	int err;
+
+	if (in->flags & ~DRM_POWERVR_SGX_SUBMIT_JOB_RENDER_FLAGS_MASK)
+		return -EINVAL;
+	if (!job->ctx->render)
+		return -EINVAL;
+	rj->rtds = rtds_lookup(to_pvrsgx_file(file), in->rt_dataset_handle);
+	if (!rj->rtds)
+		return -ENOENT;
+	if (rj->rtds->ctx != job->ctx)
+		return -EINVAL;
+	if (!in->ta_kicks.count)
+		return -EINVAL;
+	/* Scenes of more than two kicks need the middle-kick flags, not known yet. */
+	if (in->ta_kicks.count > PVRSGX_MAX_TA_KICKS)
+		return -EINVAL;
+
+	if (!rdef || !kdef)
+		return -EOPNOTSUPP;
+
+	cctx.heaps[PVRSGX_CMD_HEAP_PDS_PIXEL].base = pvrsgx_heaps(sdev)->pds_pixel.base;
+	cctx.heaps[PVRSGX_CMD_HEAP_PDS_PIXEL].size =
+		pvrsgx_heaps(sdev)->pds_pixel.end - pvrsgx_heaps(sdev)->pds_pixel.base;
+	cctx.limits[PVRSGX_CMD_LIMIT_TILES_X] = rj->rtds->n.xt - 1;
+	cctx.limits[PVRSGX_CMD_LIMIT_TILES_Y] = rj->rtds->n.yt - 1;
+	cctx.rt_pad_width = rj->rtds->n.xt * 32;
+	cctx.rt_pad_height = rj->rtds->n.yt * 16;
+	cctx.rt_samples = rj->rtds->samples;
+	cctx.mapped = pvrsgx_cmd_mapped;
+	cctx.priv = vm;
+
+	err = pvrsgx_cmd_copy(rdef, in->cmd, &rj->render, &present);
+	err = err ?: pvrsgx_cmd_take(sdev, rdef, &cctx, vm, &rj->render, present);
+	err = err ?: pvrsgx_check_render(sdev->family->cmds, &rj->render, rj->rtds->samples);
+	if (err)
+		return err;
+
+	kicks = pvrsgx_obj_array_get(&in->ta_kicks, sizeof(*kicks), PVRSGX_MAX_TA_KICKS);
+	if (IS_ERR(kicks))
+		return PTR_ERR(kicks);
+	rj->nkicks = in->ta_kicks.count;
+	for (u32 k = 0; k < rj->nkicks && !err; k++) {
+		err = pvrsgx_cmd_take_elem(kdef, &kicks[k], in->ta_kicks.stride, &rj->kick[k],
+					   &present);
+		err = err ?: pvrsgx_cmd_take(sdev, kdef, &cctx, vm, &rj->kick[k], present);
+	}
+	kvfree(kicks);
+	return err;
+}
+
+void pvrsgx_render_job_fini(struct pvrsgx_job *job)
+{
+	pvrsgx_rtds_put(job->render.rtds);
+	job->render.rtds = NULL;
+}
+
+static u32 pds_word(u64 pds, u64 addr, u32 rows)
+{
+	return rows << 26 | (u32)(addr - pds) >> 4;
+}
+
+/*
+ * Write the TA command for kick k of the job's scene (and, at the first, the
+ * render details), and say where the microkernel will report the scene done.
+ * Runs in run_job: one job on the GPU at a time, so the context's CCB and the
+ * dataset are ours.
+ *
+ * One kick both starts the scene and renders it (TA flags 0x63, the form Mesa
+ * uses). A single-draw frame can also take two kicks: 0x71 runs the stream,
+ * 0x62 runs the terminate words and starts the 3D.
+ */
+/*
+ * A restart (a reset, or a runtime resume) starts a fresh microkernel, but a
+ * VM's PB lives through it. After a hang the pages the TA took are still off
+ * its free list: the next render on any of the VM's contexts hangs, and the
+ * next restart fails. The PB's first job after a restart sets it up as new.
+ * Nothing else needs it: the HW contexts and RT datasets
+ * carry on as they are.
+ */
+static void pb_rearm(struct pvrsgx_device *sdev, struct pvrsgx_pb *pb)
+{
+	if (pb->fw_gen != sdev->fw_gen) {
+		pb_init(pb);
+		pb->fw_gen = sdev->fw_gen;
+	}
+}
+
+int pvrsgx_render_job_build(struct pvrsgx_job *job, u32 k, u32 data[6])
+{
+	struct pvrsgx_render_job *rj = &job->render;
+	struct pvrsgx_render_ctx *r = job->ctx->render;
+	struct pvrsgx_rt_dataset *s = rj->rtds;
+	const struct drm_powervr_sgx_cmd_render *in = &rj->render.base;
+	const struct drm_powervr_sgx_ext_sgx545_render *x545 = &rj->render.sgx545_render;
+	bool first = k == 0, last = k == rj->nkicks - 1;
+	struct pvrsgx_fw_ta_cmd *cmd;
+	struct pvrsgx_fw_rt_deps *deps;
+	struct pvrsgx_fw_render_details *rd;
+	u32 f, odd, p, rgn_this, rgn_next, entry, *s3d, *done, *b;
+	u32 woff = r->ccb_woff, zls_base = 0;
+	u64 pds = pvrsgx_heaps(job->ctx->sdev)->pds_pixel.base;
+
+	if (first)
+		pb_rearm(job->ctx->sdev, r->pb);
+	f = s->frames;
+	odd = f & 1;
+	/* This frame's RT-data entry and everything that goes with it. */
+	p = odd ? 0 : 1;
+	rgn_this = kobj_va(s->rgn[p]);
+	rgn_next = kobj_va(s->rgn[!p]);
+	entry = kobj_va(s->rtdata) + 0x18 + 0x38 * p;
+	rd = (struct pvrsgx_fw_render_details *)kobj_cpu(s->rd[p]);
+	if (first) {
+		/* Render details. */
+		if (in->depth_addr || in->stencil_addr) {
+			u64 lo = in->depth_addr ?: in->stencil_addr;
+
+			if (in->depth_addr && in->stencil_addr)
+				lo = min(in->depth_addr, in->stencil_addr);
+
+			zls_base = (u32)lo & 0xfff00000;
+		}
+		rd->zls_base = zls_base;
+		rd->depth_bias_units = in->depth_bias_units;
+		rd->depth_bias_factor = in->depth_bias_factor_f32;
+		rd->region_headers = rgn_this;
+		rd->region_headers_next = rgn_next;
+		rd->zls_ctl = x545->zls_control ? x545->zls_control | (s->n.zls_xt - 1) << 3 : 0;
+		rd->depth[0] = in->depth_addr ? (u32)in->depth_addr - zls_base : 0;
+		rd->depth[1] = rd->depth[0];
+		rd->stencil[0] = BIT(31) |
+				 (in->stencil_addr ? (u32)in->stencil_addr - zls_base : 0);
+		rd->stencil[1] = rd->stencil[0];
+		rd->bg_depth = in->bg_depth_f32;
+		rd->pixel_event_pds = (u32)in->pixel_event_pds_addr;
+		rd->pixel_event_rows = in->pixel_event_pds_rows;
+		rd->rt_data_entry = entry;
+		kobj_cpu(s->rd_w4[p])[0] = pds_word(pds, in->bg_pds_primary_addr,
+						    in->bg_pds_primary_rows);
+
+		/*
+		 * The background object's PDS words (secondary, middle, primary),
+		 * written by the host so the microkernel never runs on the job's
+		 * directory.
+		 */
+		b = kobj_cpu(s->bgblock[p]);
+		b[0] = pds_word(pds, in->bg_pds_secondary_addr, in->bg_pds_secondary_rows);
+		b[1] = BG_PDS_MIDDLE;
+		b[2] = kobj_cpu(s->rd_w4[p])[0];
+
+		/* What the microkernel clears and the client sets before a scene. */
+		kobj_cpu(s->rtword)[0] = f + 1;
+		kobj_cpu(s->rd_sync[p])[0] = 1;
+		kobj_cpu(s->hwsync_access[p])[0] = 1;
+	}
+
+	/* The TA command, in the context's ring. */
+	cmd = (struct pvrsgx_fw_ta_cmd *)((u8 *)kobj_cpu(r->ccb) + woff);
+	memset(cmd, 0, sizeof(*cmd));
+	cmd->size = sizeof(*cmd);
+	cmd->ta_flags = PVRSGX_TA_FLAGS_BASE | (first ? PVRSGX_TA_FIRST_KICK : 0) |
+			(last ? PVRSGX_TA_LAST_KICK : 0) |
+			(first && !f ? PVRSGX_TA_FIRST_RENDER : 0);
+	cmd->bg_pds[0] = pds_word(pds, in->bg_pds_secondary_addr, in->bg_pds_secondary_rows);
+	cmd->bg_pds[1] = BG_PDS_MIDDLE;
+	cmd->bg_pds[2] = pds_word(pds, in->bg_pds_primary_addr, in->bg_pds_primary_rows);
+	cmd->ta_ctl[0] = TA_CTL;
+	cmd->ta_ctl[1] = TA_CTL;
+	if (last)
+		cmd->flags_3d = x545->flags_3d | PVRSGX_3D_RENDER | f << PVRSGX_3D_FRAME_SHIFT;
+	cmd->frame = f;
+	cmd->pb_desc = kobj_va(r->pb->desc);
+	cmd->render_details = kobj_va(s->rd[p]);
+	cmd->rt_deps = kobj_va(s->hwsync[p]);
+	cmd->rt_data = kobj_va(s->rtdata);
+	cmd->rt_data_entry = entry;
+
+	/* Ready; the 3D sync on the last kick; the completion word on the first. */
+	cmd->ready = 1;
+	if (last) {
+		s3d = kobj_cpu(r->sync3d);
+		cmd->sync3d_write_pending = s3d[SYNC_WRITE_PENDING];
+		cmd->sync3d_write_complete = kobj_va(r->sync3d) + SYNC_WRITE_COMPLETE * 4;
+		cmd->sync3d_read_pending = s3d[SYNC_READ_PENDING]++;
+		cmd->sync3d_read_complete = kobj_va(r->sync3d) + SYNC_READ_COMPLETE * 4;
+	}
+	if (first) {
+		done = kobj_cpu(s->done);
+		deps = (struct pvrsgx_fw_rt_deps *)kobj_cpu(s->hwsync[p]);
+		deps->num_dst = 1;
+		deps->read_pending = done[SYNC_READ_PENDING];
+		deps->read_complete = kobj_va(s->done) + SYNC_READ_COMPLETE * 4;
+		deps->write_pending = done[SYNC_WRITE_PENDING]++;
+		deps->write_complete = kobj_va(s->done) + SYNC_WRITE_COMPLETE * 4;
+		/* The microkernel sets write-complete to the pending value plus one. */
+		job->done_cpu = &done[SYNC_WRITE_COMPLETE];
+		job->done_val = deps->write_pending + 1;
+	}
+
+	cmd->te_aa = s->n.te_aa;
+	cmd->te_mtile[0] = s->n.mtile1;
+	cmd->te_mtile[1] = s->n.mtile2;
+	cmd->te_screen = s->n.te_screen;
+	cmd->mt_tiles = s->n.mt_tiles;
+	cmd->ta_flags2 = rj->kick[k].sgx545_ta_kick.ta_flags2 | (last ? PVRSGX_TA2_RENDER : 0);
+	cmd->vdm_base = (u32)rj->kick[k].base.vdm_base;
+	cmd->vdm_ctl = VDM_CTL;
+	cmd->region_headers = rgn_this;
+	cmd->tail_pointers = kobj_va(s->tailptr);
+	cmd->eps[0] = F32_1E_6;
+	cmd->eps[1] = F32_1E_6;
+	cmd->screen_size = (s->h - 1) << 16 | (s->w - 1);
+	cmd->params_base = r->pb->base;
+	cmd->rt_flags = s->n.rt_flags;
+
+	r->ccb_woff = (woff + sizeof(*cmd)) & (CCB_RING - 1);
+	/* The command is in memory before the write offset that hands it over. */
+	wmb();
+	kobj_cpu(r->ccbctl)[0] = r->ccb_woff;
+	if (last) {
+		rj->cmd_off = woff;
+		rj->entry = p;
+		trace_pvrsgx_render_kick(cmd, s->w, s->h, rgn_next, woff);
+		s->frames++;
+	}
+
+	memset(data, 0, 6 * sizeof(u32));
+	data[1] = kobj_va(r->hwctx);
+	return 0;
+}
+
+/* ---------------------------------------------------------------- transfers */
+
+/* The structures of a render, word by word. */
+static void render_dump(struct seq_buf *sb, struct pvrsgx_device *sdev, struct pvrsgx_rt_dataset *s,
+			u32 cmd_off, u32 p)
+{
+	struct pvrsgx_render_ctx *r = s->ctx->render;
+
+	seq_buf_printf(sb, "PB 0x%08x pages 0x%x base 0x%08x; free list 0x%08x; scratch 0x%08x\n",
+		       kobj_va(r->pb->buf), r->pb->pages, r->pb->base, kobj_va(r->pb->freelist),
+		       kobj_va(r->scratch));
+	pvrsgx_kobj_dump(sb, "hwctx", r->hwctx, 0, sizeof(struct pvrsgx_fw_hw_render_ctx));
+	pvrsgx_kobj_dump(sb, "ccbctl", r->ccbctl, 0, 8);
+	pvrsgx_kobj_dump(sb, "pbdesc", r->pb->desc, 0, sizeof(struct pvrsgx_fw_pb_desc));
+	pvrsgx_kobj_dump(sb, "pbblocks", r->pb->blocks, 0, 0x10);
+	pvrsgx_kobj_dump(sb, "freelist[first..]", r->pb->freelist,
+			 ((kobj_va(r->pb->buf) - pvrsgx_heaps(sdev)->params_3d.base) >>
+			  PAGE_SHIFT) * 4, 0x20);
+	pvrsgx_kobj_dump(sb, "scratch", r->scratch, 0, 0x40);
+	pvrsgx_kobj_dump(sb, "sync3d", r->sync3d, 0, SYNC_DATA_BYTES);
+	pvrsgx_kobj_dump(sb, "ta_cmd", r->ccb, cmd_off, sizeof(struct pvrsgx_fw_ta_cmd));
+	seq_buf_printf(sb, "rt-data entry %u\n", p);
+	pvrsgx_kobj_dump(sb, "render_details", s->rd[p], 0,
+			 sizeof(struct pvrsgx_fw_render_details));
+	pvrsgx_kobj_dump(sb, "rd_w4", s->rd_w4[p], 0, 4);
+	pvrsgx_kobj_dump(sb, "rd_w8", s->rd_w8[p], 0, 8);
+	pvrsgx_kobj_dump(sb, "rd_sync", s->rd_sync[p], 0, 4);
+	pvrsgx_kobj_dump(sb, "rtdata", s->rtdata, 0, sizeof(struct pvrsgx_fw_rt_data));
+	pvrsgx_kobj_dump(sb, "rtword", s->rtword, 0, 4);
+	pvrsgx_kobj_dump(sb, "entword0", s->entword[0], 0, 4);
+	pvrsgx_kobj_dump(sb, "entword1", s->entword[1], 0, 4);
+	pvrsgx_kobj_dump(sb, "hwsync", s->hwsync[p], 0, sizeof(struct pvrsgx_fw_rt_deps));
+	pvrsgx_kobj_dump(sb, "hwsync_access", s->hwsync_access[p], 0, 4);
+	pvrsgx_kobj_dump(sb, "done", s->done, 0, SYNC_DATA_BYTES);
+	pvrsgx_kobj_dump(sb, "bgblock", s->bgblock[p], 0, 0x60);
+	/* what the TA wrote for the 3D */
+	pvrsgx_kobj_dump(sb, "rgn0", s->rgn[0], 0, 0x60);
+	pvrsgx_kobj_dump(sb, "rgn1", s->rgn[1], 0, 0x60);
+	pvrsgx_kobj_dump(sb, "tailptr", s->tailptr, 0, 0x40);
+	pvrsgx_kobj_dump(sb, "ta40", s->ta40, 0, 0x40);
+	pvrsgx_kobj_dump(sb, "ta50", s->ta50, 0, 0x50);
+	pvrsgx_kobj_dump(sb, "ta88[0]", s->ta88[0], 0, 0x88);
+	pvrsgx_kobj_dump(sb, "ta88[1]", s->ta88[1], 0, 0x88);
+	seq_buf_printf(sb, "rgn 0x%08x 0x%08x tailptr 0x%08x ta40 0x%08x ta50 0x%08x\n",
+		       kobj_va(s->rgn[0]), kobj_va(s->rgn[1]), kobj_va(s->tailptr),
+		       kobj_va(s->ta40), kobj_va(s->ta50));
+	seq_buf_printf(sb, "ta88 0x%08x 0x%08x\n", kobj_va(s->ta88[0]), kobj_va(s->ta88[1]));
+	pvrsgx_fw_dump(sb, sdev, PVRSGX_FW_TA3DCTL, "ta3dctl");
+}
+
+/* A job's command and the structures it names, for a devcoredump. */
+void pvrsgx_job_dump(struct seq_buf *sb, struct pvrsgx_job *job)
+{
+	struct pvrsgx_device *sdev = job->ctx->sdev;
+
+	if (job->type == DRM_POWERVR_SGX_JOB_TYPE_RENDER && job->render.rtds)
+		render_dump(sb, sdev, job->render.rtds, job->render.cmd_off, job->render.entry);
+	else if (job->type == DRM_POWERVR_SGX_JOB_TYPE_TRANSFER)
+		pvrsgx_transfer_dump(sb, job);
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_render.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_render.h
new file mode 100644
index 000000000..06a816b48
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_render.h
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_RENDER_H
+#define PVRSGX_RENDER_H
+
+#include <linux/kref.h>
+#include <linux/types.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_cmd_defs.h"
+
+struct drm_device;
+struct drm_file;
+struct pvrsgx_context;
+struct pvrsgx_device;
+struct pvrsgx_file;
+struct pvrsgx_job;
+struct pvrsgx_kobj;
+struct pvrsgx_vm;
+
+/* The firmware half of a render context. */
+/* A VM's parameter buffer, shared by its render contexts (pvrsgx_render.c). */
+struct pvrsgx_pb {
+	struct kref ref;
+	struct pvrsgx_device *sdev;
+	struct pvrsgx_kobj *buf, *freelist, *desc, *blocks;
+	u32 pages;
+	/** @base: The 3D request base, PB & 0xfff00000. */
+	u32 base;
+	/** @fw_gen: The microkernel run (pvrsgx_device.fw_gen) it was last set up for. */
+	u32 fw_gen;
+};
+
+struct pvrsgx_render_ctx {
+	struct pvrsgx_kobj *hwctx, *ccb, *ccbctl, *scratch, *sync3d;
+	struct pvrsgx_pb *pb;
+	u32 ccb_woff;
+};
+
+/* 16 MiB: the device's one PB holds every client's scenes (pvrsgx_render.c). */
+#define PVRSGX_PB_DEFAULT_PAGES	0x1000
+
+void pvrsgx_pb_put(struct pvrsgx_pb *pb);
+void pvrsgx_pb_unmap(struct pvrsgx_vm *vm);
+
+/* A render target's tiling numbers. */
+struct pvrsgx_rt_numbers {
+	u32 xt, yt, mx, my, padx, pady, zls_xt;
+	u32 mtile1, mtile2, te_screen, mt_tiles, rt_mt_tiles, region_bytes, tailptr_bytes;
+	u32 te_aa, rt_flags;
+};
+
+struct pvrsgx_rt_dataset {
+	struct kref ref;
+	struct pvrsgx_context *ctx;
+	u32 w, h, samples;
+	struct pvrsgx_rt_numbers n;
+	/*
+	 * Per RT-data entry, which alternates by frame parity, so one frame's TA
+	 * can be built while the other's 3D still reads its own: render details,
+	 * their words, the dependency block and its syncs, and the background
+	 * object, which the 3D fetches per tile.
+	 */
+	struct pvrsgx_kobj *rd[2], *rd_w8[2], *rd_w4[2], *hwsync[2];
+	struct pvrsgx_kobj *rd_sync[2], *hwsync_access[2], *bgblock[2];
+	struct pvrsgx_kobj *rtword, *rtdata, *entword[2], *done;
+	struct pvrsgx_kobj *tailptr, *ta40, *ta50, *rgn[2], *ta88[2];
+	/** @frames: Renders into this dataset so far. */
+	u32 frames;
+};
+
+/* Each context's command ring. */
+#define CCB_RING		0x10000
+
+/* A firmware sync object: pending and complete counts for writes and reads. */
+#define SYNC_WRITE_PENDING	0
+#define SYNC_WRITE_COMPLETE	1
+#define SYNC_READ_PENDING	2
+#define SYNC_READ_COMPLETE	3
+#define SYNC_DATA_BYTES		0x18
+
+#define PVRSGX_MAX_TA_KICKS	2
+
+/* What a render job carries from SUBMIT_JOBS to run_job. */
+struct pvrsgx_render_job {
+	struct pvrsgx_rt_dataset *rtds;
+	struct pvrsgx_cmd_render_all render;
+	struct pvrsgx_cmd_ta_kick_all kick[PVRSGX_MAX_TA_KICKS];
+	u32 nkicks;
+	/** @cmd_off: Where the last kick's TA command went in the context's ring. */
+	u32 cmd_off;
+	/** @entry: The RT-data entry (frame parity) the render used. */
+	u32 entry;
+};
+
+int pvrsgx_render_ctx_init(struct pvrsgx_device *sdev, struct pvrsgx_context *ctx, u32 pb_pages);
+void pvrsgx_render_ctx_fini(struct pvrsgx_device *sdev, struct pvrsgx_context *ctx);
+
+struct pvrsgx_rt_dataset *pvrsgx_rtds_create(struct pvrsgx_context *ctx, u32 w, u32 h, u32 samples);
+void pvrsgx_rtds_put(struct pvrsgx_rt_dataset *rtds);
+void pvrsgx_rtds_destroy_all(struct pvrsgx_file *sfile);
+int pvrsgx_ioctl_create_rt_dataset(struct drm_device *drm, void *data, struct drm_file *file);
+int pvrsgx_ioctl_destroy_rt_dataset(struct drm_device *drm, void *data, struct drm_file *file);
+
+int pvrsgx_render_job_init(struct drm_file *file, struct pvrsgx_job *job,
+			   const struct drm_powervr_sgx_job *in);
+void pvrsgx_render_job_fini(struct pvrsgx_job *job);
+
+int pvrsgx_render_job_build(struct pvrsgx_job *job, u32 k, u32 data[6]);
+struct seq_buf;
+void pvrsgx_job_dump(struct seq_buf *sb, struct pvrsgx_job *job);
+
+/* Command checks shared with transfer jobs. */
+bool pvrsgx_cmd_mapped(const void *priv, u64 addr, u64 size);
+const struct pvrsgx_cmd_def *pvrsgx_cmd_def(struct pvrsgx_device *sdev, enum pvrsgx_cmd_type type);
+int pvrsgx_cmd_take(struct pvrsgx_device *sdev, const struct pvrsgx_cmd_def *def,
+		    const struct pvrsgx_cmd_ctx *ctx, struct pvrsgx_vm *vm, const void *all,
+		    u64 present);
+
+#endif /* PVRSGX_RENDER_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.c
new file mode 100644
index 000000000..7a0f1a474
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.c
@@ -0,0 +1,175 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+#include <drm/drm_file.h>
+
+#include <linux/seq_buf.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_cmd.h"
+#include "pvrsgx_cmd_checks.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_fwif.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_kobj.h"
+#include "pvrsgx_mmu.h"
+#include "pvrsgx_render.h"
+#include "pvrsgx_transfer.h"
+#include "pvrsgx_vm.h"
+
+/*
+ * A transfer is a 3D-only render: the background object covers the
+ * destination, its pixel program samples the source and its end-of-tile
+ * program writes the destination, all client memory. The
+ * kernel owns the HW transfer context, its command ring and the completion
+ * sync, and writes the command from the client's CMD_TRANSFER stream.
+ */
+#define XFER_CCB_BYTES		0x104c8	/* 0x10000 ring + slack for a command that wraps */
+#define XFER_TILES_X_MAX	127	/* 4096 pixels; the real limit is not known */
+#define XFER_TILES_Y_MAX	255
+
+int pvrsgx_transfer_ctx_init(struct pvrsgx_device *sdev, struct pvrsgx_context *ctx)
+{
+	struct pvrsgx_transfer_ctx *x = kzalloc_obj(*x);
+	struct pvrsgx_fw_hw_transfer_ctx *hw;
+	int err;
+
+	if (!x)
+		return -ENOMEM;
+	ctx->transfer = x;
+
+	err = pvrsgx_kobj_new(sdev, &x->ccb, PVRSGX_KRANGE_KDATA, XFER_CCB_BYTES, 0, NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &x->ccbctl, PVRSGX_KRANGE_KDATA, 8, 0, NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &x->done, PVRSGX_KRANGE_SYNC, SYNC_DATA_BYTES, 0, NULL);
+	err = err ?: pvrsgx_kobj_new(sdev, &x->hwctx, PVRSGX_KRANGE_KCODE,
+				     sizeof(struct pvrsgx_fw_hw_transfer_ctx), 0, NULL);
+	if (err)
+		goto fail;
+
+	/* The words a transfer context needs; the rest stays zero. */
+	hw = (void *)kobj_cpu(x->hwctx);
+	hw->flags = 1;
+	hw->page_dir = pvrsgx_vm_pd_word(ctx->vm);
+	hw->ccb = kobj_va(x->ccb);
+	hw->ccb_ctl = kobj_va(x->ccbctl);
+	hw->pid = task_tgid_nr(current);
+	return 0;
+
+fail:
+	pvrsgx_transfer_ctx_fini(sdev, ctx);
+	return err;
+}
+
+void pvrsgx_transfer_ctx_fini(struct pvrsgx_device *sdev, struct pvrsgx_context *ctx)
+{
+	struct pvrsgx_transfer_ctx *x = ctx->transfer;
+
+	if (!x)
+		return;
+	pvrsgx_kobj_drop(sdev, &x->hwctx);
+	pvrsgx_kobj_drop(sdev, &x->done);
+	pvrsgx_kobj_drop(sdev, &x->ccbctl);
+	pvrsgx_kobj_drop(sdev, &x->ccb);
+	kfree(x);
+	ctx->transfer = NULL;
+}
+
+int pvrsgx_transfer_job_init(struct drm_file *file, struct pvrsgx_job *job,
+			     const struct drm_powervr_sgx_job *in)
+{
+	struct pvrsgx_device *sdev = job->ctx->sdev;
+	const struct pvrsgx_cmd_def *def = pvrsgx_cmd_def(sdev, PVRSGX_CMD_TRANSFER);
+	struct pvrsgx_transfer_job *tj = &job->transfer;
+	struct pvrsgx_vm *vm = job->ctx->vm;
+	struct pvrsgx_cmd_ctx cctx = {};
+	u64 present;
+	int err;
+
+	if (in->flags & ~DRM_POWERVR_SGX_SUBMIT_JOB_TRANSFER_FLAGS_MASK)
+		return -EINVAL;
+	if (!job->ctx->transfer || in->ta_kicks.count || in->rt_dataset_handle)
+		return -EINVAL;
+	tj->dst_twiddled = in->flags & DRM_POWERVR_SGX_SUBMIT_JOB_TRANSFER_DST_TWIDDLED;
+
+	if (!def)
+		return -EOPNOTSUPP;
+	cctx.heaps[PVRSGX_CMD_HEAP_PDS_PIXEL].base = pvrsgx_heaps(sdev)->pds_pixel.base;
+	cctx.heaps[PVRSGX_CMD_HEAP_PDS_PIXEL].size =
+		pvrsgx_heaps(sdev)->pds_pixel.end - pvrsgx_heaps(sdev)->pds_pixel.base;
+	cctx.limits[PVRSGX_CMD_LIMIT_TILES_X] = XFER_TILES_X_MAX;
+	cctx.limits[PVRSGX_CMD_LIMIT_TILES_Y] = XFER_TILES_Y_MAX;
+	cctx.mapped = pvrsgx_cmd_mapped;
+	cctx.priv = vm;
+
+	err = pvrsgx_cmd_copy(def, in->cmd, &tj->xfer, &present);
+	err = err ?: pvrsgx_cmd_take(sdev, def, &cctx, vm, &tj->xfer, present);
+	return err ?: pvrsgx_check_transfer(&tj->xfer);
+}
+
+/* Write the 0x1c0-byte transfer command into the context's ring. */
+void pvrsgx_transfer_job_build(struct pvrsgx_job *job, u32 data[6])
+{
+	struct pvrsgx_transfer_ctx *x = job->ctx->transfer;
+	const struct drm_powervr_sgx_cmd_transfer *in = &job->transfer.xfer.base;
+	u32 woff = x->ccb_woff, base = (u32)in->bg_block_addr & 0xfff00000;
+	struct pvrsgx_fw_transfer_cmd *cmd = (void *)((u8 *)kobj_cpu(x->ccb) + woff);
+	u32 *done = kobj_cpu(x->done);
+
+	job->transfer.cmd_off = woff;
+	memset(cmd, 0, sizeof(*cmd));
+	cmd->bg_object = 0x2000000 | ((u32)in->bg_block_addr - base) >> 4;
+	cmd->unk_04 = 0x200;
+	cmd->bg_depth = in->bg_depth_f32;
+	cmd->unk_0c = 3;
+	cmd->unk_14 = 0x100;
+	cmd->req_base_3d = base;
+	cmd->rt_ctl = BIT(31);
+	cmd->rt_flags = 0x88;
+	cmd->pixel_event_pds = (u32)in->pixel_event_pds_addr;
+	cmd->pixel_event_rows = in->pixel_event_pds_rows;
+	cmd->unk_2c = 1;
+	cmd->first_tile = in->first_tile_x << 16 | in->first_tile_y;
+	cmd->last_tile = in->last_tile_x << 16 | in->last_tile_y;
+	cmd->unk_38 = 0x80;
+	/* Never the client's. */
+	cmd->pds_exec_base = pvrsgx_heaps(job->ctx->sdev)->pds_pixel.base;
+	cmd->size = sizeof(*cmd);
+	cmd->flags = job->transfer.dst_twiddled ? PVRSGX_TRANSFER_DST_TWIDDLED : 0;
+	cmd->unk_194 = 0x07c00000;
+
+	/* No source syncs; one destination, the kernel's completion sync. */
+	cmd->num_dst = 1;
+	cmd->dst_read_pending = done[SYNC_READ_PENDING];
+	cmd->dst_read_complete = kobj_va(x->done) + SYNC_READ_COMPLETE * 4;
+	cmd->dst_write_pending = done[SYNC_WRITE_PENDING]++;
+	cmd->dst_write_complete = kobj_va(x->done) + SYNC_WRITE_COMPLETE * 4;
+	/* The microkernel sets write-complete to the pending value plus one. */
+	job->done_cpu = &done[SYNC_WRITE_COMPLETE];
+	job->done_val = cmd->dst_write_pending + 1;
+
+	x->ccb_woff = (woff + sizeof(*cmd)) & (CCB_RING - 1);
+	/* The command is in memory before the write offset that hands it over. */
+	wmb();
+	kobj_cpu(x->ccbctl)[0] = x->ccb_woff;
+
+	memset(data, 0, 6 * sizeof(u32));
+	data[1] = kobj_va(x->hwctx);
+}
+
+/* The transfer command and the structures it names, for a devcoredump. */
+void pvrsgx_transfer_dump(struct seq_buf *sb, struct pvrsgx_job *job)
+{
+	struct pvrsgx_transfer_ctx *x = job->ctx->transfer;
+
+	if (!x)
+		return;
+	pvrsgx_kobj_dump(sb, "xfer_hwctx", x->hwctx, 0, sizeof(struct pvrsgx_fw_hw_transfer_ctx));
+	pvrsgx_kobj_dump(sb, "xfer_ccbctl", x->ccbctl, 0, 8);
+	pvrsgx_kobj_dump(sb, "xfer_cmd", x->ccb, job->transfer.cmd_off,
+			 sizeof(struct pvrsgx_fw_transfer_cmd));
+	pvrsgx_kobj_dump(sb, "xfer_done", x->done, 0, SYNC_DATA_BYTES);
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.h
new file mode 100644
index 000000000..4e85df3ad
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_transfer.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_TRANSFER_H
+#define PVRSGX_TRANSFER_H
+
+#include <linux/types.h>
+
+#include "pvrsgx_cmd_defs.h"
+
+struct drm_file;
+struct drm_powervr_sgx_job;
+struct pvrsgx_context;
+struct pvrsgx_device;
+struct pvrsgx_job;
+struct pvrsgx_kobj;
+struct seq_buf;
+
+/* A transfer context's firmware half: HW transfer context, command ring, completion sync. */
+struct pvrsgx_transfer_ctx {
+	struct pvrsgx_kobj *hwctx, *ccb, *ccbctl, *done;
+	u32 ccb_woff;
+};
+
+struct pvrsgx_transfer_job {
+	struct pvrsgx_cmd_transfer_all xfer;
+	bool dst_twiddled;
+	u32 cmd_off;
+};
+
+int pvrsgx_transfer_ctx_init(struct pvrsgx_device *sdev, struct pvrsgx_context *ctx);
+void pvrsgx_transfer_ctx_fini(struct pvrsgx_device *sdev, struct pvrsgx_context *ctx);
+int pvrsgx_transfer_job_init(struct drm_file *file, struct pvrsgx_job *job,
+			     const struct drm_powervr_sgx_job *in);
+void pvrsgx_transfer_job_build(struct pvrsgx_job *job, u32 data[6]);
+void pvrsgx_transfer_dump(struct seq_buf *sb, struct pvrsgx_job *job);
+
+#endif /* PVRSGX_TRANSFER_H */

-- 
2.50.1

[PATCH RFC 11/13] drm/powervr_sgx: Validate shader code in the kernel

[Joseph Telaak]

A client's USSE task could read the core's registers with ldr, which no
page directory can stop. So client shader code only runs from code
objects that UPLOAD_CODE filled, and the kernel checks every 8-byte
word first: it must match a known instruction form, and it may not be
ldr, str, setl or savl, or a branch out of the object. Code objects are
mapped read-only for the GPU, and the code bases cover exactly the two
USSE heaps, so a task can run nothing else.

The table of instruction forms is generated from the ISA description.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c       | 236 ++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_usse.h       |  45 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_usse_table.h | 342 ++++++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c   | 188 +++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_validate.h   |  25 ++
 5 files changed, 836 insertions(+)

diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c
new file mode 100644
index 000000000..9aa0cd4ab
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The USSE instruction check behind UPLOAD_CODE (pvrsgx_validate.c): each
+ * word must be exactly one form of the table generated from the isaspec
+ * description, with every operand in a register file and number the form
+ * allows, and branch and phase targets inside the object. Nothing here
+ * depends on the device, so the same code also builds for a host fuzzer.
+ */
+
+#include <linux/array_size.h>
+#include <linux/bitops.h>
+#include <linux/bits.h>
+#include <linux/kconfig.h>
+
+#include "pvrsgx_usse.h"
+
+enum pvrsgx_usse_kind {
+	USSE_OK,
+	USSE_REFUSE,
+	USSE_BR,
+	USSE_PHAS,
+};
+
+enum pvrsgx_usse_slot_kind {
+	USSE_SLOT_NONE,
+	USSE_SLOT_DST,
+	USSE_SLOT_DST1,
+	USSE_SLOT_SRC,
+	USSE_SLOT_SRC0,
+};
+
+/*
+ * How an operand's number bit 6 reads: part of the number; a flag for r, o,
+ * pa, sa and i under C10 format control, which is on at program start and the
+ * only setfc allowed keeps so; or, for every register file, a flag while
+ * format select (that bit of the word) is set.
+ */
+#define USSE_NUM	0
+#define USSE_C10	1
+#define USSE_SEL(bit)	(0x80 | (bit))
+
+#define USSE_MAX_SLOTS	4
+#define USSE_MAX_PIECES	3
+#define USSE_ALWAYS	0xff
+
+/* An operand slot: its value is the pieces' bits, each shifted into place. */
+struct pvrsgx_usse_slot {
+	u8 kind;
+	/* USSE_NUM, USSE_C10 or USSE_SEL(). */
+	u8 fmt;
+	/* The slot exists only while this bit of the word is set. */
+	u8 when;
+	struct {
+		u8 lo, width, shift;
+	} p[USSE_MAX_PIECES];
+};
+
+struct pvrsgx_usse_form {
+	u64 mask, match;
+	u8 kind;
+	u8 slots;
+};
+
+#include "pvrsgx_usse_table.h"
+
+/* The forms of each instruction set (pvrsgx_core.h). */
+static const struct {
+	const struct pvrsgx_usse_form *forms;
+	u32 n;
+} isas[] = {
+	[PVRSGX_ISA_SGX545] = { pvrsgx_usse_forms, ARRAY_SIZE(pvrsgx_usse_forms) },
+};
+
+/* The register files an operand's bank and extension bits select. */
+enum usse_file {
+	USSE_R,		/* temporaries */
+	USSE_O,		/* outputs */
+	USSE_PA,	/* primary attributes */
+	USSE_SA,	/* secondary attributes */
+	USSE_INDEXED,	/* through idx1 or idx2 */
+	USSE_CG,	/* constants c0-63, then globals g0- */
+	USSE_IDX,	/* the index registers */
+	USSE_IMM,	/* an immediate */
+	USSE_I,		/* internal registers */
+};
+
+/* By bank | extension << 2 (dst, src) or << 1 (src0, dst1). */
+static const u8 dst_files[8] = {
+	USSE_R, USSE_O, USSE_PA, USSE_INDEXED, USSE_SA, USSE_CG, USSE_IDX, USSE_I,
+};
+
+static const u8 src_files[8] = {
+	USSE_R, USSE_O, USSE_PA, USSE_SA, USSE_INDEXED, USSE_CG, USSE_IMM, USSE_I,
+};
+
+static const u8 src0_files[4] = { USSE_R, USSE_PA, USSE_O, USSE_SA };
+
+/* g46 on (number 110) is undocumented; i0-i3 and idx1-idx2 are what exists. */
+#define USSE_CG_END		110
+#define USSE_I_LAST		3
+
+static bool usse_slot_ok(const struct pvrsgx_usse_slot *s, u64 w)
+{
+	bool sel = (s->fmt & 0x80) && (w & BIT_ULL(s->fmt & 0x3f));
+	u32 v = 0, n, file;
+
+	if (s->when != USSE_ALWAYS && !(w & BIT_ULL(s->when)))
+		return true;
+	for (int i = 0; i < USSE_MAX_PIECES && s->p[i].width; i++)
+		v |= ((w >> s->p[i].lo) & GENMASK_ULL(s->p[i].width - 1, 0)) << s->p[i].shift;
+	n = v & 0x7f;
+	switch (s->kind) {
+	case USSE_SLOT_DST:
+		file = dst_files[(v >> 7) & 7];
+		break;
+	case USSE_SLOT_SRC:
+		file = src_files[(v >> 7) & 7];
+		break;
+	case USSE_SLOT_SRC0:
+		file = src0_files[(v >> 7) & 3];
+		break;
+	case USSE_SLOT_DST1:
+		file = (v >> 7) & 1 ? USSE_PA : USSE_R;
+		break;
+	default:
+		return false;
+	}
+	/* Bit 6 is a flag only where the format says so; elsewhere it counts. */
+	if (sel || (s->fmt == USSE_C10 && file == USSE_I))
+		n &= 0x3f;
+	switch (file) {
+	case USSE_CG:
+		/* Read only: what writing c or g does isn't known. */
+		return s->kind != USSE_SLOT_DST && n < USSE_CG_END;
+	case USSE_I:
+		return n <= USSE_I_LAST;
+	case USSE_IDX:
+		return n == 1 || n == 2;
+	default:
+		return true;
+	}
+}
+
+/* br's and phas.i's targets: l0-17, in instructions. */
+#define USSE_TARGET		GENMASK_ULL(17, 0)
+
+static s64 refuse(enum pvrsgx_usse_refusal *why, enum pvrsgx_usse_refusal r, u32 i)
+{
+	if (why)
+		*why = r;
+	return i;
+}
+
+/**
+ * pvrsgx_usse_scan() - Check USSE code before it goes into a code object
+ * @isa: The core's instruction set.
+ * @w: The instructions.
+ * @n: Number of @w.
+ * @first: Instruction index @w[0] will sit at in the object.
+ * @obj_insts: The object's size in instructions.
+ * @place: The object's start, in instructions from the start of the code base
+ *	window it is mapped in, or %PVRSGX_USSE_PLACE_NONE. An absolute target
+ *	(phas.i) needs it, and an object with one must lie in one window.
+ * @abs: Set when an instruction has an absolute target.
+ * @why: If not NULL, why the instruction returned was refused.
+ *
+ * Linear in @n, no allocation.
+ *
+ * Return: the index in @w of the first refused instruction, or -1.
+ */
+s64 pvrsgx_usse_scan(enum pvrsgx_isa isa, const u64 *w, u32 n, u32 first, u32 obj_insts,
+		     s64 place, bool *abs, enum pvrsgx_usse_refusal *why)
+{
+	const struct pvrsgx_usse_form *forms = isas[isa].forms;
+
+	if (why)
+		*why = PVRSGX_USSE_REFUSAL_NONE;
+	for (u32 i = 0; i < n; i++) {
+		const struct pvrsgx_usse_form *f = NULL;
+		s64 target;
+
+		for (u32 k = 0; k < isas[isa].n; k++) {
+			if ((w[i] & forms[k].mask) == forms[k].match) {
+				f = &forms[k];
+				break;
+			}
+		}
+		if (!f)
+			return refuse(why, PVRSGX_USSE_REFUSAL_FORM, i);
+		for (int k = 0; k < USSE_MAX_SLOTS; k++) {
+			const struct pvrsgx_usse_slot *s = &pvrsgx_usse_slots[f->slots][k];
+
+			if (s->kind != USSE_SLOT_NONE && !usse_slot_ok(s, w[i]))
+				return refuse(why, PVRSGX_USSE_REFUSAL_OPERAND, i);
+		}
+		switch (f->kind) {
+		case USSE_OK:
+			break;
+		case USSE_BR:
+			target = (s64)first + i + sign_extend64(w[i] & USSE_TARGET, 17);
+			if (target < 0 || target >= obj_insts)
+				return refuse(why, PVRSGX_USSE_REFUSAL_BRANCH, i);
+			break;
+		case USSE_PHAS:
+			/* 0: no next phase. Otherwise from the task's code base. */
+			target = w[i] & USSE_TARGET;
+			if (!target)
+				break;
+			if (place == PVRSGX_USSE_PLACE_NONE || target < place ||
+			    target >= place + obj_insts ||
+			    place + obj_insts > (s64)PVRSGX_USSE_WINDOW_INSTS)
+				return refuse(why, PVRSGX_USSE_REFUSAL_PHAS, i);
+			*abs = true;
+			break;
+		default:
+			return refuse(why, PVRSGX_USSE_REFUSAL_KIND, i);
+		}
+	}
+	return -1;
+}
+
+const char *pvrsgx_usse_refusal_name(enum pvrsgx_usse_refusal why)
+{
+	static const char *const names[] = {
+		[PVRSGX_USSE_REFUSAL_NONE] = "none",
+		[PVRSGX_USSE_REFUSAL_FORM] = "no known form",
+		[PVRSGX_USSE_REFUSAL_OPERAND] = "operand",
+		[PVRSGX_USSE_REFUSAL_KIND] = "refused instruction",
+		[PVRSGX_USSE_REFUSAL_BRANCH] = "branch target outside the object",
+		[PVRSGX_USSE_REFUSAL_PHAS] = "phase target outside the object",
+	};
+
+	return why < ARRAY_SIZE(names) ? names[why] : "?";
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.h
new file mode 100644
index 000000000..c63b883c6
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_USSE_H
+#define PVRSGX_USSE_H
+
+#include <linux/bits.h>
+#include <linux/limits.h>
+#include <linux/types.h>
+
+#include "pvrsgx_core.h"
+
+/* A code object's place when it isn't mapped, or is mapped at more than one. */
+#define PVRSGX_USSE_PLACE_NONE	S64_MIN
+
+/*
+ * One code base's window, in instructions: 512 KiB. A task's absolute targets
+ * (phas.i) count from the base its PDS program names, which a compiler sets to
+ * the window of the task's first word.
+ */
+#define PVRSGX_USSE_WINDOW_INSTS	BIT(16)
+
+/* Why pvrsgx_usse_scan() refused an instruction. */
+enum pvrsgx_usse_refusal {
+	PVRSGX_USSE_REFUSAL_NONE,
+	/* Not exactly one known form. */
+	PVRSGX_USSE_REFUSAL_FORM,
+	/* An operand's register file or number. */
+	PVRSGX_USSE_REFUSAL_OPERAND,
+	/* A form that is refused outright. */
+	PVRSGX_USSE_REFUSAL_KIND,
+	/* br's target outside the object. */
+	PVRSGX_USSE_REFUSAL_BRANCH,
+	/*
+	 * phas.i's target outside the object, the object has no one place, or it
+	 * crosses a code base window.
+	 */
+	PVRSGX_USSE_REFUSAL_PHAS,
+};
+
+s64 pvrsgx_usse_scan(enum pvrsgx_isa isa, const u64 *w, u32 n, u32 first, u32 obj_insts,
+		     s64 place, bool *abs, enum pvrsgx_usse_refusal *why);
+const char *pvrsgx_usse_refusal_name(enum pvrsgx_usse_refusal why);
+
+#endif /* PVRSGX_USSE_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_usse_table.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_usse_table.h
new file mode 100644
index 000000000..1a0b09063
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_usse_table.h
@@ -0,0 +1,342 @@
+/* SPDX-License-Identifier: MIT */
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The USSE forms, their operand slots and what the validator does with
+ * each, generated by gen_usse_table.py from sgx.xml, the isaspec
+ * description of the instruction set Mesa's compiler uses; both live in
+ * the driver's source repository, https://github.com/The1TrueJoe/powervr_sgx.
+ * Each form is the bits its patterns fix; each slot the fields its operand
+ * is built from.
+ */
+
+static const struct pvrsgx_usse_slot pvrsgx_usse_slots[][USSE_MAX_SLOTS] = {
+	[0] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC0, USSE_SEL(54), USSE_ALWAYS, { { 14, 7, 0 }, { 34, 1, 7 } } },
+		{ USSE_SLOT_SRC, USSE_SEL(54), USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_SEL(54), USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[1] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_SEL(54), USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_SEL(54), USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[2] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+	},
+	[3] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS, { { 21, 7, 0 }, { 32, 2, 7 } } },
+		{ USSE_SLOT_SRC0, USSE_SEL(54), USSE_ALWAYS, { { 14, 7, 0 }, { 34, 1, 7 } } },
+		{ USSE_SLOT_SRC, USSE_SEL(54), USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_SEL(54), USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[4] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC0, USSE_NUM, USSE_ALWAYS, { { 14, 7, 0 }, { 34, 1, 7 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[5] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[6] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS, { { 21, 7, 0 }, { 32, 2, 7 } } },
+		{ USSE_SLOT_SRC0, USSE_NUM, USSE_ALWAYS,
+		  { { 14, 7, 0 }, { 34, 1, 7 }, { 38, 1, 8 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS, { { 7, 7, 0 }, { 30, 2, 7 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 39, 1, 9 } } },
+	},
+	[7] = {
+		{ USSE_SLOT_DST, USSE_SEL(50), 20, { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_SEL(50), USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_SEL(50), USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[8] = {
+		{ USSE_SLOT_DST, USSE_C10, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_C10, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_C10, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[9] = {
+		{ USSE_SLOT_DST, USSE_C10, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC0, USSE_C10, USSE_ALWAYS, { { 14, 7, 0 }, { 34, 1, 7 } } },
+		{ USSE_SLOT_SRC, USSE_C10, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_C10, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[10] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS, { { 14, 7, 0 }, { 30, 2, 7 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS, { { 7, 7, 0 }, { 30, 2, 7 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[11] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[12] = {
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 14, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[13] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 14, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[14] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+		{ USSE_SLOT_SRC0, USSE_NUM, USSE_ALWAYS,
+		  { { 14, 7, 0 }, { 34, 1, 7 }, { 47, 1, 8 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[15] = {
+		{ USSE_SLOT_DST1, USSE_NUM, USSE_ALWAYS, { { 21, 7, 0 }, { 39, 1, 7 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[16] = {
+		{ USSE_SLOT_SRC0, USSE_NUM, USSE_ALWAYS,
+		  { { 14, 7, 0 }, { 34, 1, 7 }, { 50, 1, 8 } } },
+		{ USSE_SLOT_DST1, USSE_NUM, USSE_ALWAYS, { { 21, 7, 0 }, { 39, 1, 7 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[17] = {
+		{ USSE_SLOT_DST1, USSE_NUM, USSE_ALWAYS, { { 21, 7, 0 }, { 36, 1, 7 } } },
+		{ USSE_SLOT_SRC0, USSE_NUM, USSE_ALWAYS, { { 14, 7, 0 }, { 34, 1, 7 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS, { { 7, 7, 0 }, { 30, 2, 7 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS, { { 0, 7, 0 }, { 28, 2, 7 } } },
+	},
+	[18] = {
+		{ USSE_SLOT_SRC0, USSE_NUM, USSE_ALWAYS,
+		  { { 14, 7, 0 }, { 34, 1, 7 }, { 50, 1, 8 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[19] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS, { { 21, 7, 0 }, { 32, 2, 7 } } },
+		{ USSE_SLOT_SRC0, USSE_NUM, USSE_ALWAYS,
+		  { { 14, 7, 0 }, { 34, 1, 7 }, { 51, 1, 8 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[20] = { },
+	[21] = {
+		{ USSE_SLOT_SRC0, USSE_NUM, USSE_ALWAYS,
+		  { { 14, 7, 0 }, { 34, 1, 7 }, { 51, 1, 8 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[22] = {
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+		{ USSE_SLOT_DST1, USSE_NUM, USSE_ALWAYS, { { 21, 7, 0 }, { 39, 1, 7 } } },
+	},
+	[23] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS,
+		  { { 21, 7, 0 }, { 32, 2, 7 }, { 51, 1, 9 } } },
+	},
+	[24] = {
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+	},
+	[25] = {
+		{ USSE_SLOT_DST, USSE_NUM, USSE_ALWAYS, { { 21, 7, 0 }, { 32, 2, 7 } } },
+	},
+	[26] = {
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+	},
+	[27] = {
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 0, 7, 0 }, { 28, 2, 7 }, { 48, 1, 9 } } },
+		{ USSE_SLOT_SRC, USSE_NUM, USSE_ALWAYS,
+		  { { 7, 7, 0 }, { 30, 2, 7 }, { 49, 1, 9 } } },
+	},
+};
+
+/* 109 forms, 126 entries */
+static const struct pvrsgx_usse_form pvrsgx_usse_forms[] = {
+	{ 0xf800060000000000ull, 0x0000020000000000ull, USSE_OK, 0 },	/* fadm */
+	{ 0xf800060000000000ull, 0x0000000000000000ull, USSE_OK, 0 },	/* fmad */
+	{ 0xf800060000000000ull, 0x0000040000000000ull, USSE_OK, 0 },	/* fmsa */
+	{ 0xf8000684001fc000ull, 0x0000060000000000ull, USSE_OK, 1 },	/* fsubflr */
+	{ 0xf8410600301fc07full, 0x0801040000000000ull, USSE_OK, 2 },	/* fcos */
+	{ 0xf8410600301fc07full, 0x0800060000000000ull, USSE_OK, 2 },	/* fexp2 */
+	{ 0xf8410600301fc07full, 0x0800040000000000ull, USSE_OK, 2 },	/* flog2 */
+	{ 0xf8410600301fc07full, 0x0800000000000000ull, USSE_OK, 2 },	/* frcp */
+	{ 0xf8410600301fc07full, 0x0800020000000000ull, USSE_OK, 2 },	/* frsq */
+	{ 0xf8410600301fc07full, 0x0801020000000000ull, USSE_OK, 2 },	/* fsin */
+	{ 0xf8410600301fc07full, 0x0801000000000000ull, USSE_OK, 2 },	/* fsqrt */
+	{ 0xf800060000000000ull, 0x1000020000000000ull, USSE_OK, 3 },	/* fddp */
+	{ 0xf800060000000000ull, 0x1000040000000000ull, USSE_OK, 3 },	/* fddpc */
+	{ 0xf8000684001fc000ull, 0x1000000000000000ull, USSE_OK, 1 },	/* fdp */
+	{ 0xf8000684001c4000ull, 0x1000000000004000ull, USSE_OK, 1 },	/* fdpc */
+	{ 0xf800060000000000ull, 0x1800000000000000ull, USSE_OK, 0 },	/* fmaxmin */
+	{ 0xf800060000000000ull, 0x1800020000000000ull, USSE_OK, 0 },	/* fminmax */
+	{ 0xf8000684001fc000ull, 0x2000000000000000ull, USSE_OK, 1 },	/* fdsx */
+	{ 0xf8000684001fc000ull, 0x2000020000000000ull, USSE_OK, 1 },	/* fdsy */
+	{ 0xf84107fc301fc07full, 0x2801000020000000ull, USSE_OK, 2 },	/* mov */
+	{ 0xf800077800000000ull, 0x2800050000000000ull, USSE_OK, 4 },	/* movc.c10 */
+	{ 0xf800077800000000ull, 0x2800040000000000ull, USSE_OK, 4 },	/* movc.f32 */
+	{ 0xf800077800000000ull, 0x2800020000000000ull, USSE_OK, 4 },	/* movc.i16 */
+	{ 0xf800077800000000ull, 0x2800030000000000ull, USSE_OK, 4 },	/* movc.i32 */
+	{ 0xf800077800000000ull, 0x2800010000000000ull, USSE_OK, 4 },	/* movc.u8 */
+	{ 0xf800071c001fc000ull, 0x2800060000000000ull, USSE_REFUSE, 5 },	/* op0xf3.s6 */
+	{ 0xf800071800000000ull, 0x2800070000000000ull, USSE_REFUSE, 4 },	/* op0xf3.s7 */
+	{ 0xf800000000000000ull, 0x3000000000000000ull, USSE_OK, 4 },	/* fmad16 */
+	{ 0xf800000000000000ull, 0x3800000000000000ull, USSE_OK, 6 },	/* efo */
+	{ 0xf800000000000000ull, 0x4000000000000000ull, USSE_OK, 5 },	/* pck */
+	{ 0xf800000000000000ull, 0x4800000000000000ull, USSE_OK, 7 },	/* test */
+	{ 0xf800000800000000ull, 0x5000000000000000ull, USSE_OK, 5 },	/* and */
+	{ 0xf800000800000000ull, 0x5000000800000000ull, USSE_OK, 5 },	/* or */
+	{ 0xf800000800000000ull, 0x5800000000000000ull, USSE_OK, 5 },	/* xor */
+	{ 0xf800000800000000ull, 0x6000000800000000ull, USSE_OK, 5 },	/* rol */
+	{ 0xf800000800000000ull, 0x6000000000000000ull, USSE_OK, 5 },	/* shl */
+	{ 0xf800000800000000ull, 0x6800000800000000ull, USSE_OK, 5 },	/* asr */
+	{ 0xf800000800000000ull, 0x6800000000000000ull, USSE_OK, 5 },	/* shr */
+	{ 0xf8010ff8301f007full, 0x7001000020000000ull, USSE_OK, 2 },	/* rlp */
+	{ 0xf800004c00000000ull, 0x7800000000000000ull, USSE_OK, 7 },	/* testmask */
+	{ 0xf800000000000000ull, 0x8000000000000000ull, USSE_OK, 8 },	/* sop2 */
+	{ 0xf820040000000000ull, 0x8820000000000000ull, USSE_OK, 9 },	/* lrp1 */
+	{ 0xf820040000000000ull, 0x8800000000000000ull, USSE_OK, 9 },	/* sop3 */
+	{ 0xf820040000000000ull, 0x8800040000000000ull, USSE_OK, 9 },	/* sop3l */
+	{ 0xf820040000000000ull, 0x8820040000000000ull, USSE_OK, 9 },	/* sop3x */
+	{ 0xf8000004001fc000ull, 0x9000000000000000ull, USSE_OK, 8 },	/* sopwm */
+	{ 0xf800000000000000ull, 0x9800000000000000ull, USSE_OK, 9 },	/* ima8 */
+	{ 0xf800800000000000ull, 0xa000000000000000ull, USSE_OK, 4 },	/* ima16 */
+	{ 0xf800000000000000ull, 0xa800000000000000ull, USSE_OK, 4 },	/* imae */
+	{ 0xf9308ffc001fc000ull, 0xb000000000000000ull, USSE_OK, 5 },	/* adif */
+	{ 0xf9300ff800000000ull, 0xb000004000000000ull, USSE_OK, 4 },	/* adifsum */
+	{ 0xf83000f800000000ull, 0xb020000000000000ull, USSE_OK, 4 },	/* bilin */
+	{ 0xf930400000000000ull, 0xb010000000000000ull, USSE_REFUSE, 4 },	/* op16s1 */
+	{ 0xf8320b7c00000000ull, 0xb030000000000000ull, USSE_OK, 10 },	/* ssum16 */
+	{ 0xf8320b7c00000000ull, 0xb030020000000000ull, USSE_OK, 10 },	/* ssum16.o8 */
+	{ 0xf8320b7c00000000ull, 0xb030010000000000ull, USSE_OK, 10 },	/* ssum16.s8 */
+	{ 0xf8320b7cc0003f80ull, 0xb030030000000000ull, USSE_REFUSE, 11 },	/* ssum16i */
+	{ 0xf800800400000000ull, 0xb800000400000000ull, USSE_REFUSE, 12 },	/* firh */
+	{ 0xf800000400000000ull, 0xb800000000000000ull, USSE_OK, 13 },	/* firv */
+	{ 0xf9000020001c4000ull, 0xc000000000000000ull, USSE_OK, 8 },	/* dot3 */
+	{ 0xf900002000000000ull, 0xc000002000000000ull, USSE_OK, 9 },	/* dot3s */
+	{ 0xf9000020001c4000ull, 0xc100000000000000ull, USSE_OK, 8 },	/* dot4 */
+	{ 0xf900002000000000ull, 0xc100002000000000ull, USSE_OK, 9 },	/* dot4s */
+	{ 0xf800000000000000ull, 0xc800000000000000ull, USSE_OK, 9 },	/* fpma */
+	{ 0xf930000800000000ull, 0xd000000000000000ull, USSE_OK, 14 },	/* ima32 */
+	{ 0xf9388f7e001fc000ull, 0xd010000000000000ull, USSE_REFUSE, 15 },	/* op1adrc */
+	{ 0xf800000000000000ull, 0xd800000000000000ull, USSE_REFUSE, 3 },	/* fdual */
+	{ 0xf808000200000000ull, 0xe000000000000000ull, USSE_OK, 16 },	/* smp */
+	{ 0xf840000800000000ull, 0xe840000000000000ull, USSE_OK, 17 },	/* eld.d */
+	{ 0xf840000800000000ull, 0xe840000800000000ull, USSE_OK, 17 },	/* eld.q */
+	{ 0xf840000000000000ull, 0xe800000000000000ull, USSE_OK, 16 },	/* ld */
+	{ 0xf80000800fe00000ull, 0xf000000000000000ull, USSE_OK, 18 },	/* st */
+	{ 0xff30f00000000000ull, 0xf930000000000000ull, USSE_OK, 19 },	/* atst8 */
+	{ 0xf87fe1fffffc0000ull, 0xf800000000000000ull, USSE_REFUSE, 20 },	/* ba */
+	{ 0xff7fe3fffffc0000ull, 0xf800004000000000ull, USSE_BR, 20 },	/* br */
+	{ 0xff7fe3fffffc0000ull, 0xf900004000000000ull, USSE_BR, 20 },	/* br */
+	{ 0xff7fe3fffffc0000ull, 0xfa00004000000000ull, USSE_BR, 20 },	/* br */
+	{ 0xff7fe3fffffc0000ull, 0xfb00004000000000ull, USSE_BR, 20 },	/* br */
+	{ 0xff7fe3fffffc0000ull, 0xfc00004000000000ull, USSE_BR, 20 },	/* br */
+	{ 0xff7fe3fffffc0000ull, 0xfd00004000000000ull, USSE_BR, 20 },	/* br */
+	{ 0xff7fe3fffffc0000ull, 0xfe00004000000000ull, USSE_BR, 20 },	/* br */
+	{ 0xff70f1cb0fe00000ull, 0xfb30000000000000ull, USSE_OK, 21 },	/* depthf */
+	{ 0xff30000000000000ull, 0xfb20000000000000ull, USSE_OK, 21 },	/* emit */
+	{ 0xffffb7feffffffffull, 0xf820000000000000ull, USSE_OK, 20 },	/* idf */
+	{ 0xfffb000000000000ull, 0xfc10000000000000ull, USSE_REFUSE, 20 },	/* imo */
+	{ 0xf87fe7ffffffffffull, 0xf800008000000000ull, USSE_REFUSE, 20 },	/* lapc */
+	{ 0xff7ef77ec0003f80ull, 0xfe20000000000000ull, USSE_REFUSE, 22 },	/* ldr */
+	{ 0xff30000cf0000000ull, 0xfc20000000000000ull, USSE_OK, 23 },	/* limm */
+	{ 0xfffff7ffffffff8full, 0xfd20000000000000ull, USSE_REFUSE, 20 },	/* lock */
+	{ 0xff7bf7fffffffffcull, 0xf800014000000000ull, USSE_OK, 20 },	/* nop */
+	{ 0xfffc34ff0fffc000ull, 0xf840300000000000ull, USSE_REFUSE, 24 },	/* op0x100 */
+	{ 0xff7fe7ffffffffffull, 0xf808020000000000ull, USSE_REFUSE, 20 },	/* op0x98 */
+	{ 0xfffff7ffffffffffull, 0xff20000000000000ull, USSE_OK, 20 },	/* op0xd6 */
+	{ 0xfffff7fffffffff0ull, 0xfa20000000000000ull, USSE_REFUSE, 20 },	/* op0xd7 */
+	{ 0xff7387fcf01fffffull, 0xfe10000000000000ull, USSE_REFUSE, 23 },	/* op0xfd */
+	{ 0xfffc34ff0fffc000ull, 0xf840200000000000ull, USSE_REFUSE, 24 },	/* op0xfe */
+	{ 0xfffc34ff0fffc000ull, 0xf840100000000000ull, USSE_REFUSE, 24 },	/* op0xff */
+	{ 0xff70f7c30fe00000ull, 0xf830000000000000ull, USSE_OK, 21 },	/* pcoeff */
+	{ 0xfff4f7ff0fffc000ull, 0xfa40000000000000ull, USSE_REFUSE, 24 },	/* phas */
+	{ 0xfff7df00fffc0000ull, 0xfa44000000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44010000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44020000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44070000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44080000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44090000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa440a0000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa440f0000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44100000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44110000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44120000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xfff7df00fffc0000ull, 0xfa44170000000000ull, USSE_PHAS, 20 },	/* phas.i */
+	{ 0xff7bf7ffffffffffull, 0xf830800000000000ull, USSE_OK, 20 },	/* ptoff */
+	{ 0xfffff7ffffffff8full, 0xfd20000000000001ull, USSE_REFUSE, 20 },	/* release */
+	{ 0xf87fe7fcf01fffffull, 0xf800010000000000ull, USSE_REFUSE, 25 },	/* savl */
+	{ 0xfffbffffffffffffull, 0xfd10000000000100ull, USSE_OK, 20 },	/* setfc */
+	{ 0xf87de7ff3fffc07full, 0xf80000c000000000ull, USSE_REFUSE, 26 },	/* setl */
+	{ 0xfffb000000000000ull, 0xfb10000000000000ull, USSE_REFUSE, 20 },	/* smbo */
+	{ 0xfff8ffff0fffc000ull, 0xfb18000000000000ull, USSE_REFUSE, 24 },	/* smbo.r */
+	{ 0xfffb000000000000ull, 0xfa10000000000000ull, USSE_OK, 20 },	/* smlsi */
+	{ 0xfff8ffff0fffc000ull, 0xfa18000000000000ull, USSE_REFUSE, 24 },	/* smlsi.r */
+	{ 0xfffb000000000000ull, 0xf810000000000000ull, USSE_REFUSE, 20 },	/* smoa */
+	{ 0xfff8ffff0fffc000ull, 0xf818000000000000ull, USSE_REFUSE, 24 },	/* smoa.r */
+	{ 0xfffb000000000000ull, 0xf910000000000000ull, USSE_REFUSE, 20 },	/* smr */
+	{ 0xfff8ffff0fffc000ull, 0xf918000000000000ull, USSE_REFUSE, 24 },	/* smr.r */
+	{ 0xff7cf7ff0fe00000ull, 0xfe28000000000000ull, USSE_REFUSE, 27 },	/* str */
+	{ 0xfffff7feffffffffull, 0xf920000000000000ull, USSE_OK, 20 },	/* wdf */
+};
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c
new file mode 100644
index 000000000..cf9cfcf87
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c
@@ -0,0 +1,188 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Shader validation: what a client's USSE code may contain. The threat model
+ * is in Documentation/gpu/powervr_sgx/index.rst, "Shader validation".
+ *
+ * Client USSE code runs only from the two USSE heaps of its own VM (the code
+ * base registers cover exactly those), the USSE heaps take only code objects,
+ * and a code object's contents arrive only through UPLOAD_CODE, which checks
+ * every instruction against the form table generated from the isaspec
+ * description: its form, what the form does, and each operand's register file
+ * and number.
+ */
+
+#include <drm/drm_file.h>
+#include <drm/drm_gem.h>
+#include <drm/drm_gpuvm.h>
+#include <drm/drm_print.h>
+
+#include <linux/dma-resv.h>
+#include <linux/iosys-map.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_core.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_gem.h"
+#include "pvrsgx_usse.h"
+#include "pvrsgx_validate.h"
+
+static bool bo_is_code(struct pvrsgx_bo *bo)
+{
+	return bo->flags & DRM_POWERVR_SGX_BO_CODE;
+}
+
+static bool heap_is_usse(unsigned int heap)
+{
+	return heap == DRM_POWERVR_SGX_HEAP_USSE_PIXEL || heap == DRM_POWERVR_SGX_HEAP_USSE_VERTEX;
+}
+
+/* Code objects, and only they, go into the USSE heaps, read-only. */
+int pvrsgx_validate_vm_map(struct pvrsgx_device *sdev, struct pvrsgx_vm *vm,
+			   unsigned int heap, struct pvrsgx_bo *bo, u64 offset, u64 size)
+{
+	return heap_is_usse(heap) == bo_is_code(bo) ? 0 : -EINVAL;
+}
+
+static_assert(PVRSGX_USSE_WINDOW_INSTS * 8 == 1u << PVRSGX_USSE_WINDOW_LOG2);
+
+/*
+ * Where a mapping at addr of the object from offset puts the object's start,
+ * in instructions from the start of its code base window: what phas.i's
+ * targets count from when the task's PDS program names the window its first
+ * word is in. One window per object makes that base the same for every entry.
+ */
+static s64 code_place(const struct pvrsgx_heap_layout *h, u64 addr, u64 offset)
+{
+	u64 base = addr >= h->usse_vertex.base ? h->usse_vertex.base : h->usse_pixel.base;
+	s64 start = (s64)(addr - base) - (s64)offset;
+
+	if (start < 0)
+		return PVRSGX_USSE_PLACE_NONE;
+	return (start / 8) & (PVRSGX_USSE_WINDOW_INSTS - 1);
+}
+
+/*
+ * Once code with absolute targets is in an object, every mapping of it puts it
+ * where those targets were checked. Called with the object's reservation held.
+ */
+int pvrsgx_validate_vm_link(struct pvrsgx_bo *bo, u64 addr, u64 offset)
+{
+	if (!bo_is_code(bo) || !bo->code_abs)
+		return 0;
+	const struct pvrsgx_heap_layout *h = pvrsgx_heaps(to_pvrsgx_device(bo->base.base.dev));
+
+	return code_place(h, addr, offset) == bo->code_place ? 0 : -EINVAL;
+}
+
+/* The one place every mapping of obj puts it, or PVRSGX_USSE_PLACE_NONE. */
+static s64 code_place_mapped(struct drm_gem_object *obj)
+{
+	const struct pvrsgx_heap_layout *h = pvrsgx_heaps(to_pvrsgx_device(obj->dev));
+	struct drm_gpuvm_bo *vm_bo;
+	struct drm_gpuva *va;
+	s64 place = PVRSGX_USSE_PLACE_NONE;
+
+	drm_gem_for_each_gpuvm_bo(vm_bo, obj) {
+		drm_gpuvm_bo_for_each_va(va, vm_bo) {
+			s64 p = code_place(h, va->va.addr, va->gem.offset);
+
+			if (place != PVRSGX_USSE_PLACE_NONE && p != place)
+				return PVRSGX_USSE_PLACE_NONE;
+			place = p;
+		}
+	}
+	return place;
+}
+
+u32 pvrsgx_validate_pte_flags(struct pvrsgx_bo *bo)
+{
+	return bo_is_code(bo) ? SGX_MMU_PTE_READONLY : 0;
+}
+
+int pvrsgx_validate_job(struct pvrsgx_device *sdev, struct pvrsgx_job *job)
+{
+	/* A job can only reach validated USSE code: nothing to check per job. */
+	return 0;
+}
+
+int pvrsgx_ioctl_upload_code(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_upload_code_args *args = data;
+	struct pvrsgx_device *sdev = to_pvrsgx_device(drm);
+	struct drm_gem_object *obj;
+	struct pvrsgx_bo *bo;
+	struct iosys_map map;
+	u64 end, *buf;
+	enum pvrsgx_usse_refusal why;
+	s64 bad, place;
+	bool abs = false;
+	int err;
+
+	args->bad_offset = ~0ull;
+	if (args->flags || !args->size || !IS_ALIGNED(args->size, 8) ||
+	    !IS_ALIGNED(args->offset, 8) || check_add_overflow(args->offset, args->size, &end))
+		return -EINVAL;
+
+	obj = drm_gem_object_lookup(file, args->handle);
+	if (!obj)
+		return -ENOENT;
+	bo = to_pvrsgx_bo(obj);
+	if (!bo_is_code(bo) || end > obj->size) {
+		err = -EINVAL;
+		goto out_obj;
+	}
+
+	buf = kvmalloc(args->size, GFP_KERNEL);
+	if (!buf) {
+		err = -ENOMEM;
+		goto out_obj;
+	}
+	if (copy_from_user(buf, u64_to_user_ptr(args->pointer), args->size)) {
+		err = -EFAULT;
+		goto out_buf;
+	}
+	/* VM_BIND links a mapping under the same lock. */
+	err = dma_resv_lock_interruptible(obj->resv, NULL);
+	if (err)
+		goto out_buf;
+	/* An unmapped mapping counts until its jobs and the BIF invalidate are done. */
+	if ((!list_empty(&obj->gpuva.list) || atomic_read(&bo->code_unmapping)) &&
+	    args->offset < bo->code_end) {
+		err = -EBUSY;
+		goto out_unlock;
+	}
+	place = code_place_mapped(obj);
+	bad = pvrsgx_usse_scan(sdev->family->isa, buf, args->size / 8, args->offset / 8,
+			       obj->size / 8, place, &abs, &why);
+	if (bad >= 0) {
+		args->bad_offset = args->offset + bad * 8;
+		drm_dbg_driver(drm, "UPLOAD_CODE: word at 0x%llx refused: %s\n",
+			       args->bad_offset, pvrsgx_usse_refusal_name(why));
+		err = -EINVAL;
+		goto out_unlock;
+	}
+	err = drm_gem_shmem_vmap_locked(&bo->base, &map);
+	if (err)
+		goto out_unlock;
+	iosys_map_memcpy_to(&map, args->offset, buf, args->size);
+	drm_gem_shmem_vunmap_locked(&bo->base, &map);
+	bo->code_end = max(bo->code_end, end);
+	if (abs) {
+		bo->code_abs = true;
+		bo->code_place = place;
+	}
+
+out_unlock:
+	dma_resv_unlock(obj->resv);
+out_buf:
+	kvfree(buf);
+out_obj:
+	drm_gem_object_put(obj);
+	return err;
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.h
new file mode 100644
index 000000000..e4871c181
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_VALIDATE_H
+#define PVRSGX_VALIDATE_H
+
+#include <linux/types.h>
+
+struct pvrsgx_bo;
+struct pvrsgx_device;
+struct pvrsgx_job;
+struct pvrsgx_vm;
+
+struct drm_device;
+struct drm_file;
+
+/* The shader validator (Documentation/gpu/powervr_sgx/index.rst, "Shader validation"). */
+int pvrsgx_validate_vm_map(struct pvrsgx_device *sdev, struct pvrsgx_vm *vm,
+			   unsigned int heap, struct pvrsgx_bo *bo, u64 offset, u64 size);
+int pvrsgx_validate_vm_link(struct pvrsgx_bo *bo, u64 addr, u64 offset);
+u32 pvrsgx_validate_pte_flags(struct pvrsgx_bo *bo);
+int pvrsgx_validate_job(struct pvrsgx_device *sdev, struct pvrsgx_job *job);
+int pvrsgx_ioctl_upload_code(struct drm_device *drm, void *data, struct drm_file *file);
+
+#endif /* PVRSGX_VALIDATE_H */

-- 
2.50.1

[PATCH RFC 12/13] drm/powervr_sgx: Add the DRM driver, power management and debugging

[Joseph Telaak]

Tie the parts together and build them.

- The DRM driver: the ioctls of the uAPI patch, render node only, on the
  SGX545 of the Intel Atom CE5300 (a PCI function, 8086:089b) and of
  Cedarview (GMA3600/3650, the auxiliary device gma500 publishes).
- The OMAP3630's SGX530 through the img,powervr-sgx device tree binding
  (DRM_POWERVR_SGX_OF), for bring-up only. It has one memory context,
  so the driver refuses it unless powervr_sgx.allow_unisolated=1 is set,
  and with it set clients aren't isolated from the firmware or from each
  other. It isn't for general use.
- Runtime PM: the microkernel powers off when the GPU is idle. System
  sleep drains the GPU, powers the microkernel off and starts it again
  after resume.
- fdinfo: each file's render-job time and its objects' memory.
- debugfs status and control blocks, and a devcoredump when a job hangs.
- The doorbell, the fault check, the reset and debugfs run under
  drm_dev_enter(), because behind gma500 the register mapping goes away
  with the auxiliary device at unbind.
- The Kconfig entry, which builds the driver.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/Kconfig                      |   1 +
 drivers/gpu/drm/Makefile                     |   1 +
 drivers/gpu/drm/powervr_sgx/Kconfig          |  49 ++++
 drivers/gpu/drm/powervr_sgx/Makefile         |  34 +++
 drivers/gpu/drm/powervr_sgx/pvrsgx_aux.c     |  83 ++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.c |  78 ++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.h |  13 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_drv.c     | 378 +++++++++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_drv.h     |  52 ++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_dump.c    | 227 ++++++++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_dump.h    |  16 ++
 drivers/gpu/drm/powervr_sgx/pvrsgx_of.c      | 119 +++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_pm.c      | 138 ++++++++++
 drivers/gpu/drm/powervr_sgx/pvrsgx_pm.h      |  30 +++
 14 files changed, 1219 insertions(+)

diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
index 61111d50b..35217ddf8 100644
--- a/drivers/gpu/drm/Kconfig
+++ b/drivers/gpu/drm/Kconfig
@@ -320,6 +320,7 @@ source "drivers/gpu/drm/hisilicon/Kconfig"
 source "drivers/gpu/drm/hyperv/Kconfig"
 source "drivers/gpu/drm/i915/Kconfig"
 source "drivers/gpu/drm/imagination/Kconfig"
+source "drivers/gpu/drm/powervr_sgx/Kconfig"
 source "drivers/gpu/drm/imx/Kconfig"
 source "drivers/gpu/drm/ingenic/Kconfig"
 source "drivers/gpu/drm/kmb/Kconfig"
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 2c6bc093a..d4e275b6f 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -237,6 +237,7 @@ obj-y			+= solomon/
 obj-$(CONFIG_DRM_SPRD) += sprd/
 obj-$(CONFIG_DRM_LOONGSON) += loongson/
 obj-$(CONFIG_DRM_POWERVR) += imagination/
+obj-$(CONFIG_DRM_POWERVR_SGX) += powervr_sgx/
 obj-$(CONFIG_DRM_VERISILICON_DC) += verisilicon/
 
 # Ensure drm headers are self-contained and pass kernel-doc
diff --git a/drivers/gpu/drm/powervr_sgx/Kconfig b/drivers/gpu/drm/powervr_sgx/Kconfig
new file mode 100644
index 000000000..dbbba1d68
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/Kconfig
@@ -0,0 +1,49 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright © 2026 Joseph Telaak
+
+config DRM_POWERVR_SGX
+	tristate "PowerVR SGX5xx (SGX545: Intel CE5300, Cedarview)"
+	depends on DRM && PCI && MMU && (X86 || ARM || COMPILE_TEST)
+	select DRM_EXEC
+	select DRM_GEM_SHMEM_HELPER
+	select DRM_GPUVM
+	select DRM_SCHED
+	select CRC32
+	select FW_LOADER
+	select WANT_DEV_COREDUMP
+	help
+	  Render-only DRM driver for PowerVR SGX5xx GPUs. It supports the
+	  SGX545 of the Intel Atom CE5300 media processors, where the GPU is
+	  its own PCI function (8086:089b), and of Cedarview (GMA3600/3650,
+	  with DRM_POWERVR_SGX_GMA500). The GPU runs a microkernel loaded
+	  with request_firmware().
+
+	  Not compatible with the out-of-tree PowerVR services driver
+	  (SGX545_CE): bind one or the other.
+
+	  If M is selected the module will be called powervr_sgx.
+
+config DRM_POWERVR_SGX_GMA500
+	bool "Cedarview SGX545, behind gma500"
+	depends on DRM_POWERVR_SGX && DRM_GMA500
+	select AUXILIARY_BUS
+	help
+	  Bind to the SGX545 of the Atom D2000/N2000 (Cedarview) GPU. There
+	  the SGX shares a PCI function with the display, which gma500
+	  drives; gma500 publishes the SGX as an auxiliary device, hands it
+	  the register mapping and forwards its interrupt.
+
+	  Say Y if this kernel runs on a Cedarview machine.
+
+config DRM_POWERVR_SGX_OF
+	bool "SGX on device tree SoCs (TI OMAP3630 SGX530)"
+	depends on DRM_POWERVR_SGX && OF
+	help
+	  Bind to an SGX described by the img,powervr-sgx device tree
+	  binding. Only parts whose GPU node sits in a ti-sysc module, which
+	  clocks and powers it, are handled: the OMAP3630's SGX530. Its
+	  clients aren't isolated from the firmware or from each other, so
+	  the driver runs it only with powervr_sgx.allow_unisolated=1, for
+	  bring-up. Dma-buf import isn't supported on ARM yet.
+
+	  Say N unless this kernel is for such an SoC.
diff --git a/drivers/gpu/drm/powervr_sgx/Makefile b/drivers/gpu/drm/powervr_sgx/Makefile
new file mode 100644
index 000000000..b441281ca
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/Makefile
@@ -0,0 +1,34 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright © 2026 Joseph Telaak
+
+CFLAGS_pvrsgx_trace.o := -I$(src)
+
+powervr_sgx-y := \
+	pvrsgx_ccb.o \
+	pvrsgx_core.o \
+	pvrsgx_device.o \
+	pvrsgx_drv.o \
+	pvrsgx_dump.o \
+	pvrsgx_fw.o \
+	pvrsgx_gem.o \
+	pvrsgx_job.o \
+	pvrsgx_kobj.o \
+	pvrsgx_mmu.o \
+	pvrsgx_pds.o \
+	pvrsgx_prime.o \
+	pvrsgx_render.o \
+	pvrsgx_transfer.o \
+	pvrsgx_trace.o \
+	pvrsgx_usse.o \
+	pvrsgx_validate.o \
+	pvrsgx_vm.o \
+	pvrsgx_cmd_checks.o \
+	pvrsgx_cmd.o \
+	pvrsgx_cmd_defs.o
+
+powervr_sgx-$(CONFIG_DEBUG_FS) += pvrsgx_debugfs.o
+powervr_sgx-$(CONFIG_PM) += pvrsgx_pm.o
+powervr_sgx-$(CONFIG_DRM_POWERVR_SGX_GMA500) += pvrsgx_aux.o
+powervr_sgx-$(CONFIG_DRM_POWERVR_SGX_OF) += pvrsgx_of.o
+
+obj-$(CONFIG_DRM_POWERVR_SGX) += powervr_sgx.o
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_aux.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_aux.c
new file mode 100644
index 000000000..61648497d
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_aux.c
@@ -0,0 +1,83 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The SGX545 of a Cedarview GPU (GMA3600/3650). It shares the PCI function and
+ * register BAR with the display engine, which gma500 drives; gma500 publishes
+ * the SGX as the auxiliary device "gma500_gfx.sgx" with its register window mapped
+ * and hands over its interrupt; it never touches the SGX itself
+ * (include/drm/gma500_sgx.h).
+ */
+
+#include <linux/auxiliary_bus.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/pm_runtime.h>
+
+#include <drm/gma500_sgx.h>
+
+#include "pvrsgx_device.h"
+#include "pvrsgx_pm.h"
+#include "pvrsgx_drv.h"
+
+static int pvrsgx_aux_probe(struct auxiliary_device *adev, const struct auxiliary_device_id *id)
+{
+	struct gma500_sgx_device *gma = to_gma500_sgx_device(adev);
+	struct device *dev = &adev->dev;
+	struct pvrsgx_device *sdev;
+	int err;
+
+	if (!gma->regs || gma->regs_len < PVRSGX_REGS_MIN || !dev_is_pci(dev->parent)) {
+		dev_err(dev, "unexpected SGX register window (%pa bytes); not touching it\n",
+			&gma->regs_len);
+		return -ENODEV;
+	}
+
+	sdev = pvrsgx_alloc(dev);
+	if (IS_ERR(sdev))
+		return PTR_ERR(sdev);
+	/* The SGX does DMA as the GPU's PCI function: no device of its own. */
+	drm_dev_set_dma_dev(&sdev->base, dev->parent);
+	sdev->gma_sgx = gma;
+	sdev->family = pvrsgx_core_family_get(gma->core);
+	sdev->regs = gma->regs;
+	sdev->irq = to_pci_dev(dev->parent)->irq;
+	auxiliary_set_drvdata(adev, sdev);
+
+	/* The graphics island is up only while gma500 is. */
+	err = pm_runtime_resume_and_get(dev->parent);
+	if (err)
+		return err;
+	err = pvrsgx_probe_common(sdev);
+	pm_runtime_put(dev->parent);
+	return err;
+}
+
+static void pvrsgx_aux_remove(struct auxiliary_device *adev)
+{
+	pvrsgx_remove_common(auxiliary_get_drvdata(adev));
+}
+
+static const struct auxiliary_device_id pvrsgx_aux_ids[] = {
+	{ .name = "gma500_gfx.sgx" },
+	{ }
+};
+MODULE_DEVICE_TABLE(auxiliary, pvrsgx_aux_ids);
+
+static struct auxiliary_driver pvrsgx_aux_driver = {
+	.name = "sgx",
+	.probe = pvrsgx_aux_probe,
+	.remove = pvrsgx_aux_remove,
+	.id_table = pvrsgx_aux_ids,
+	.driver.pm = pm_ptr(&pvrsgx_pm_ops),
+};
+
+int pvrsgx_aux_register(void)
+{
+	return auxiliary_driver_register(&pvrsgx_aux_driver);
+}
+
+void pvrsgx_aux_unregister(void)
+{
+	auxiliary_driver_unregister(&pvrsgx_aux_driver);
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.c
new file mode 100644
index 000000000..99f1b7a8c
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.c
@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+#include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_file.h>
+#include <drm/drm_print.h>
+
+#include <linux/debugfs.h>
+#include <linux/ktime.h>
+#include <linux/pm_runtime.h>
+#include <linux/seq_buf.h>
+#include <linux/seq_file.h>
+#include <linux/slab.h>
+
+#include "pvrsgx_debugfs.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_dump.h"
+#include "pvrsgx_fw.h"
+
+/*
+ * The registers only while the device is still bound and already awake:
+ * reading never resumes the GPU, and a suspend can't start under the read.
+ */
+static int status_show(struct seq_file *m, void *unused)
+{
+	struct drm_debugfs_entry *entry = m->private;
+	struct pvrsgx_device *sdev = to_pvrsgx_device(entry->dev);
+	struct seq_buf sb;
+	char *buf = kmalloc(SZ_4K, GFP_KERNEL);
+	int idx, awake;
+
+	if (!buf)
+		return -ENOMEM;
+	if (!drm_dev_enter(entry->dev, &idx)) {
+		kfree(buf);
+		return -ENODEV;
+	}
+	/* -EINVAL: runtime PM is off and the device always on. */
+	awake = pm_runtime_get_if_active(sdev->base.dev);
+	seq_buf_init(&sb, buf, SZ_4K);
+	pvrsgx_dump_status(&sb, sdev, awake > 0 || awake == -EINVAL);
+	if (awake > 0)
+		pm_runtime_put(sdev->base.dev);
+	drm_dev_exit(idx);
+	seq_puts(m, seq_buf_str(&sb));
+	kfree(buf);
+	return 0;
+}
+
+/* The microkernel's control blocks, raw, with a timestamp: memory only, so safe to poll. */
+static int ctlblk_show(struct seq_file *m, void *unused)
+{
+	struct drm_debugfs_entry *entry = m->private;
+	struct pvrsgx_device *sdev = to_pvrsgx_device(entry->dev);
+	struct seq_buf sb;
+	char *buf = kmalloc(SZ_2K, GFP_KERNEL);
+
+	if (!buf)
+		return -ENOMEM;
+	seq_buf_init(&sb, buf, SZ_2K);
+	seq_buf_printf(&sb, "t %lld gen %u\n", ktime_get_ns(), READ_ONCE(sdev->fw_gen));
+	pvrsgx_fw_dump(&sb, sdev, PVRSGX_FW_TA3DCTL, "ta3dctl");
+	pvrsgx_fw_dump(&sb, sdev, PVRSGX_FW_HOSTCTL, "hostctl");
+	seq_puts(m, seq_buf_str(&sb));
+	kfree(buf);
+	return 0;
+}
+
+static const struct drm_debugfs_info pvrsgx_debugfs_list[] = {
+	{ "status", status_show, 0 },
+	{ "ctlblk", ctlblk_show, 0 },
+};
+
+void pvrsgx_debugfs_init(struct drm_minor *minor)
+{
+	drm_debugfs_add_files(minor->dev, pvrsgx_debugfs_list, ARRAY_SIZE(pvrsgx_debugfs_list));
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.h
new file mode 100644
index 000000000..9bdc9a43c
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_debugfs.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_DEBUGFS_H
+#define PVRSGX_DEBUGFS_H
+
+struct drm_minor;
+
+#if defined(CONFIG_DEBUG_FS)
+void pvrsgx_debugfs_init(struct drm_minor *minor);
+#endif
+
+#endif /* PVRSGX_DEBUGFS_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_drv.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_drv.c
new file mode 100644
index 000000000..fe996a17f
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_drv.c
@@ -0,0 +1,378 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+#include <drm/drm_drv.h>
+#include <drm/drm_file.h>
+#include <drm/drm_gem.h>
+#include <drm/drm_gem_shmem_helper.h>
+#include <drm/drm_ioctl.h>
+#include <drm/drm_prime.h>
+#include <drm/drm_print.h>
+
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_debugfs.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_drv.h"
+#include "pvrsgx_pm.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_gem.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_prime.h"
+#include "pvrsgx_render.h"
+#include "pvrsgx_validate.h"
+#include "pvrsgx_vm.h"
+
+/* Elements are copied at min(stride, obj_size); a longer input tail must be zero. */
+void *pvrsgx_obj_array_get(const struct drm_powervr_sgx_obj_array *in, u32 obj_size,
+			   u32 max_count)
+{
+	u8 __user *src = u64_to_user_ptr(in->array);
+	u8 *out;
+
+	if (!in->count)
+		return NULL;
+	if (!in->stride || in->count > max_count)
+		return ERR_PTR(-EINVAL);
+
+	out = kvcalloc(in->count, obj_size, GFP_KERNEL);
+	if (!out)
+		return ERR_PTR(-ENOMEM);
+	for (u32 i = 0; i < in->count; i++) {
+		int err = copy_struct_from_user(out + (size_t)i * obj_size, obj_size,
+						src + (size_t)i * in->stride, in->stride);
+
+		if (err) {
+			kvfree(out);
+			/* -E2BIG: a larger struct this kernel doesn't know. */
+			return ERR_PTR(err);
+		}
+	}
+	return out;
+}
+
+int pvrsgx_obj_array_put(const struct drm_powervr_sgx_obj_array *out, const void *src,
+			 u32 obj_size, u32 count)
+{
+	u8 __user *dst = u64_to_user_ptr(out->array);
+	u32 n = min(out->count, count), len = min(out->stride, obj_size);
+
+	if (!out->stride)
+		return -EINVAL;
+	for (u32 i = 0; i < n; i++) {
+		if (copy_to_user(dst + (size_t)i * out->stride,
+				 (const u8 *)src + (size_t)i * obj_size, len))
+			return -EFAULT;
+		if (out->stride > len &&
+		    clear_user(dst + (size_t)i * out->stride + len, out->stride - len))
+			return -EFAULT;
+	}
+	return 0;
+}
+
+/* Copy a fixed-size query result: min of both sizes out, the kernel's size back. */
+static int query_put(void __user *ptr, u32 *size, const void *val, u32 len)
+{
+	if (ptr && copy_to_user(ptr, val, min(*size, len)))
+		return -EFAULT;
+	*size = ptr ? min(*size, len) : len;
+	return 0;
+}
+
+static int pvrsgx_ioctl_dev_query(struct drm_device *drm, void *data, struct drm_file *file)
+{
+	struct drm_powervr_sgx_ioctl_dev_query_args *args = data;
+	struct pvrsgx_device *sdev = to_pvrsgx_device(drm);
+	void __user *ptr = u64_to_user_ptr(args->pointer);
+
+	switch (args->type) {
+	case DRM_POWERVR_SGX_DEV_QUERY_GPU_INFO_GET: {
+		struct drm_powervr_sgx_dev_query_gpu_info info = {
+			.core_id = sdev->core_id,
+			.core_revision = sdev->core_rev,
+			.fw_interface_version = sdev->fw.interface,
+		};
+
+		return query_put(ptr, &args->size, &info, sizeof(info));
+	}
+	case DRM_POWERVR_SGX_DEV_QUERY_LIMITS_GET: {
+		/*
+		 * Render-target limits are not documented: these are the sizes
+		 * known to work, to be raised when larger targets are tested.
+		 */
+		struct drm_powervr_sgx_dev_query_limits lim = {
+			.rt_max_width = 2048,
+			.rt_max_height = 2048,
+			.rt_max_samples = sdev->family->cmds->max_samples,
+			.pb_min_pages = 0x100,
+			.pb_default_pages = PVRSGX_PB_DEFAULT_PAGES,
+			.pb_max_pages = pvrsgx_heaps(sdev)->pb_max_pages,
+			.max_ta_kicks = PVRSGX_MAX_TA_KICKS,
+			.va_bits = sdev->family->va_bits,
+			.caps = DRM_POWERVR_SGX_CAP_BACKWARD_BLITS,
+			.max_layers = 1,
+		};
+
+		return query_put(ptr, &args->size, &lim, sizeof(lim));
+	}
+	case DRM_POWERVR_SGX_DEV_QUERY_QUIRKS_GET: {
+		struct drm_powervr_sgx_dev_query_quirks q = {};
+		int err;
+
+		if (ptr) {
+			err = copy_struct_from_user(&q, sizeof(q), ptr, args->size);
+			if (err)
+				return err;
+			if (q._padding_c)
+				return -EINVAL;
+		}
+		/* No erratum that reaches userspace is known for these cores. */
+		q.count = 0;
+		q.musthave_count = 0;
+		return query_put(ptr, &args->size, &q, sizeof(q));
+	}
+	case DRM_POWERVR_SGX_DEV_QUERY_HEAP_INFO_GET:
+		return pvrsgx_query_heap_info(sdev, ptr, &args->size);
+	case DRM_POWERVR_SGX_DEV_QUERY_COMMANDS_GET: {
+		const struct pvrsgx_cmd_def *render = pvrsgx_cmd_def(sdev, PVRSGX_CMD_RENDER);
+		const struct pvrsgx_cmd_def *kick = pvrsgx_cmd_def(sdev, PVRSGX_CMD_TA_KICK);
+		const struct pvrsgx_cmd_def *xfer = pvrsgx_cmd_def(sdev, PVRSGX_CMD_TRANSFER);
+		struct drm_powervr_sgx_dev_query_commands c = {
+			.job_types = BIT(DRM_POWERVR_SGX_JOB_TYPE_NULL),
+		};
+
+		/* A render needs its kicks too; an unsupported command is refused. */
+		if (render && render->supported && kick && kick->supported)
+			c.job_types |= BIT(DRM_POWERVR_SGX_JOB_TYPE_RENDER);
+		if (xfer && xfer->supported)
+			c.job_types |= BIT(DRM_POWERVR_SGX_JOB_TYPE_TRANSFER);
+		c.render_exts = render ? pvrsgx_cmd_ext_mask(render) : 0;
+		c.ta_kick_exts = kick ? pvrsgx_cmd_ext_mask(kick) : 0;
+		c.transfer_exts = xfer ? pvrsgx_cmd_ext_mask(xfer) : 0;
+		return query_put(ptr, &args->size, &c, sizeof(c));
+	}
+	default:
+		return -EINVAL;
+	}
+}
+
+static int pvrsgx_open(struct drm_device *drm, struct drm_file *file)
+{
+	struct pvrsgx_file *sfile = kzalloc_obj(*sfile);
+
+	if (!sfile)
+		return -ENOMEM;
+	sfile->usage = pvrsgx_usage_new();
+	if (!sfile->usage) {
+		kfree(sfile);
+		return -ENOMEM;
+	}
+	sfile->sdev = to_pvrsgx_device(drm);
+	xa_init_flags(&sfile->vms, XA_FLAGS_ALLOC1);
+	xa_init_flags(&sfile->contexts, XA_FLAGS_ALLOC1);
+	xa_init_flags(&sfile->rtds, XA_FLAGS_ALLOC1);
+	file->driver_priv = sfile;
+	return 0;
+}
+
+static void pvrsgx_postclose(struct drm_device *drm, struct drm_file *file)
+{
+	struct pvrsgx_file *sfile = to_pvrsgx_file(file);
+
+	pvrsgx_rtds_destroy_all(sfile);
+	pvrsgx_context_destroy_all(sfile);
+	pvrsgx_vm_destroy_all(sfile);
+	xa_destroy(&sfile->rtds);
+	xa_destroy(&sfile->contexts);
+	xa_destroy(&sfile->vms);
+	pvrsgx_usage_put(sfile->usage);
+	kfree(sfile);
+	file->driver_priv = NULL;
+}
+
+#define IOCTL(n, f) DRM_IOCTL_DEF_DRV(POWERVR_SGX_##n, f, DRM_RENDER_ALLOW)
+
+static const struct drm_ioctl_desc pvrsgx_ioctls[] = {
+	IOCTL(DEV_QUERY, pvrsgx_ioctl_dev_query),
+	IOCTL(CREATE_BO, pvrsgx_ioctl_create_bo),
+	IOCTL(GET_BO_MMAP_OFFSET, pvrsgx_ioctl_get_bo_mmap_offset),
+	IOCTL(CREATE_VM_CONTEXT, pvrsgx_ioctl_create_vm_context),
+	IOCTL(DESTROY_VM_CONTEXT, pvrsgx_ioctl_destroy_vm_context),
+	IOCTL(VM_BIND, pvrsgx_ioctl_vm_bind),
+	IOCTL(CREATE_CONTEXT, pvrsgx_ioctl_create_context),
+	IOCTL(DESTROY_CONTEXT, pvrsgx_ioctl_destroy_context),
+	IOCTL(GET_CONTEXT_STATE, pvrsgx_ioctl_get_context_state),
+	IOCTL(CREATE_RT_DATASET, pvrsgx_ioctl_create_rt_dataset),
+	IOCTL(DESTROY_RT_DATASET, pvrsgx_ioctl_destroy_rt_dataset),
+	IOCTL(SUBMIT_JOBS, pvrsgx_ioctl_submit_jobs),
+	IOCTL(UPLOAD_CODE, pvrsgx_ioctl_upload_code),
+};
+
+/* drm-usage-stats.rst: the file's render time and its objects' memory. */
+static void pvrsgx_show_fdinfo(struct drm_printer *p, struct drm_file *file)
+{
+	struct pvrsgx_file *sfile = to_pvrsgx_file(file);
+
+	drm_printf(p, "drm-engine-render:\t%llu ns\n",
+		   (u64)atomic64_read(&sfile->usage->render_ns));
+	drm_show_memory_stats(p, file);
+}
+
+static const struct file_operations pvrsgx_fops = {
+	.owner = THIS_MODULE,
+	DRM_GEM_FOPS,
+	.show_fdinfo = drm_show_fdinfo,
+};
+
+static const struct drm_driver pvrsgx_drm_driver = {
+	.driver_features = DRIVER_GEM | DRIVER_RENDER | DRIVER_SYNCOBJ | DRIVER_SYNCOBJ_TIMELINE,
+	.open = pvrsgx_open,
+	.postclose = pvrsgx_postclose,
+	.ioctls = pvrsgx_ioctls,
+	.num_ioctls = ARRAY_SIZE(pvrsgx_ioctls),
+	.fops = &pvrsgx_fops,
+	.show_fdinfo = pvrsgx_show_fdinfo,
+#if defined(CONFIG_DEBUG_FS)
+	.debugfs_init = pvrsgx_debugfs_init,
+#endif
+	.gem_create_object = pvrsgx_gem_create_object,
+	.gem_prime_import_sg_table = pvrsgx_prime_import_sg_table,
+
+	.name = PVRSGX_DRIVER_NAME,
+	.desc = PVRSGX_DRIVER_DESC,
+	.major = PVRSGX_DRIVER_MAJOR,
+	.minor = PVRSGX_DRIVER_MINOR,
+};
+
+struct pvrsgx_device *pvrsgx_alloc(struct device *dev)
+{
+	return devm_drm_dev_alloc(dev, &pvrsgx_drm_driver, struct pvrsgx_device, base);
+}
+
+/* What both buses do once the registers are mapped and the interrupt source known. */
+int pvrsgx_probe_common(struct pvrsgx_device *sdev)
+{
+	int err;
+
+	err = pvrsgx_device_init(sdev);
+	if (err)
+		return err;
+
+	/* Runtime PM is set up before userspace can reach the device. */
+	pvrsgx_power_init(sdev);
+	err = drm_dev_register(&sdev->base, 0);
+	if (err) {
+		pvrsgx_power_fini(sdev);
+		pvrsgx_device_unplug(sdev);
+	}
+	return err;
+}
+
+void pvrsgx_remove_common(struct pvrsgx_device *sdev)
+{
+	/* No new ioctl, and the running ones done, before the hardware goes. */
+	drm_dev_unplug(&sdev->base);
+	pvrsgx_power_fini(sdev);
+	pvrsgx_device_unplug(sdev);
+}
+
+static int pvrsgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+	struct pvrsgx_device *sdev;
+	int err;
+
+	err = pcim_enable_device(pdev);
+	if (err)
+		return err;
+	pci_set_master(pdev);
+	err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+	if (err)
+		return err;
+	/* The GPU's MMU maps page by page: any segment length will do. */
+	dma_set_max_seg_size(&pdev->dev, UINT_MAX);
+
+	sdev = pvrsgx_alloc(&pdev->dev);
+	if (IS_ERR(sdev))
+		return PTR_ERR(sdev);
+	sdev->pdev = pdev;
+	sdev->family = pvrsgx_core_family_get(id->driver_data);
+	sdev->irq = pdev->irq;
+	pci_set_drvdata(pdev, sdev);
+
+	if (pci_resource_len(pdev, 0) < PVRSGX_REGS_MIN)
+		return -ENODEV;
+	sdev->regs = pcim_iomap_region(pdev, 0, PVRSGX_DRIVER_NAME);
+	if (IS_ERR(sdev->regs))
+		return PTR_ERR(sdev->regs);
+
+	return pvrsgx_probe_common(sdev);
+}
+
+static void pvrsgx_remove(struct pci_dev *pdev)
+{
+	pvrsgx_remove_common(pci_get_drvdata(pdev));
+}
+
+static void pvrsgx_shutdown(struct pci_dev *pdev)
+{
+	struct pvrsgx_device *sdev = pci_get_drvdata(pdev);
+
+	/* Leave the GPU quiet for whatever runs next. */
+	pvrsgx_fw_stop(sdev);
+}
+
+static const struct pci_device_id pvrsgx_pci_ids[] = {
+	{ PCI_VDEVICE(INTEL, 0x089b), 545 },	/* CE5300 family: SGX545 */
+	{ }
+};
+MODULE_DEVICE_TABLE(pci, pvrsgx_pci_ids);
+
+static struct pci_driver pvrsgx_pci_driver = {
+	.name = PVRSGX_DRIVER_NAME,
+	.id_table = pvrsgx_pci_ids,
+	.probe = pvrsgx_probe,
+	.remove = pvrsgx_remove,
+	.shutdown = pvrsgx_shutdown,
+	.driver.pm = pm_ptr(&pvrsgx_pm_ops),
+};
+
+static int __init pvrsgx_init(void)
+{
+	int err = pci_register_driver(&pvrsgx_pci_driver);
+
+	if (err)
+		return err;
+	err = pvrsgx_aux_register();
+	if (err)
+		goto err_pci;
+	err = pvrsgx_of_register();
+	if (err)
+		goto err_aux;
+	return 0;
+
+err_aux:
+	pvrsgx_aux_unregister();
+err_pci:
+	pci_unregister_driver(&pvrsgx_pci_driver);
+	return err;
+}
+module_init(pvrsgx_init);
+
+static void __exit pvrsgx_exit(void)
+{
+	pvrsgx_of_unregister();
+	pvrsgx_aux_unregister();
+	pci_unregister_driver(&pvrsgx_pci_driver);
+}
+module_exit(pvrsgx_exit);
+
+MODULE_AUTHOR("Joseph Telaak <joseph@telaak.dev>");
+MODULE_DESCRIPTION("PowerVR SGX5xx DRM driver");
+MODULE_LICENSE("GPL");
+MODULE_IMPORT_NS("DMA_BUF");
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_drv.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_drv.h
new file mode 100644
index 000000000..a6acc5234
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_drv.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_DRV_H
+#define PVRSGX_DRV_H
+
+#include <linux/types.h>
+
+#define PVRSGX_DRIVER_NAME	"powervr_sgx"
+#define PVRSGX_DRIVER_DESC	"PowerVR SGX5xx"
+#define PVRSGX_DRIVER_MAJOR	1
+#define PVRSGX_DRIVER_MINOR	0
+
+#define PVRSGX_BO_FLAGS_MASK		(DRM_POWERVR_SGX_BO_ALLOW_CPU_ACCESS | \
+					 DRM_POWERVR_SGX_BO_CODE)
+#define PVRSGX_SYNC_OP_FLAGS_MASK	(DRM_POWERVR_SGX_SYNC_OP_FLAG_HANDLE_TYPE_MASK | \
+					 DRM_POWERVR_SGX_SYNC_OP_FLAG_SIGNAL)
+
+/* The registers this driver touches all lie below this offset. */
+#define PVRSGX_REGS_MIN			0x4000
+
+struct device;
+struct dev_pm_ops;
+struct drm_powervr_sgx_obj_array;
+struct pvrsgx_device;
+
+struct pvrsgx_device *pvrsgx_alloc(struct device *dev);
+int pvrsgx_probe_common(struct pvrsgx_device *sdev);
+void pvrsgx_remove_common(struct pvrsgx_device *sdev);
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_GMA500)
+int pvrsgx_aux_register(void);
+void pvrsgx_aux_unregister(void);
+#else
+static inline int pvrsgx_aux_register(void) { return 0; }
+static inline void pvrsgx_aux_unregister(void) { }
+#endif
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_OF)
+int pvrsgx_of_register(void);
+void pvrsgx_of_unregister(void);
+#else
+static inline int pvrsgx_of_register(void) { return 0; }
+static inline void pvrsgx_of_unregister(void) { }
+#endif
+
+void *pvrsgx_obj_array_get(const struct drm_powervr_sgx_obj_array *in, u32 obj_size,
+			   u32 max_count);
+int pvrsgx_obj_array_put(const struct drm_powervr_sgx_obj_array *out, const void *src,
+			 u32 obj_size, u32 count);
+
+#endif /* PVRSGX_DRV_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_dump.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_dump.c
new file mode 100644
index 000000000..e37d48432
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_dump.c
@@ -0,0 +1,227 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * Device state as text: debugfs "status", and the devcoredump taken when a
+ * job hangs (/sys/class/devcoredump/devcd<N>/data; write to it to free it).
+ */
+
+#include <drm/drm_print.h>
+
+#include <linux/bitfield.h>
+#include <linux/devcoredump.h>
+#include <linux/seq_buf.h>
+#include <linux/vmalloc.h>
+
+#include <uapi/drm/powervr_sgx_drm.h>
+
+#include "pvrsgx_ccb.h"
+#include "pvrsgx_device.h"
+#include "pvrsgx_dump.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_kobj.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_mmu.h"
+#include "pvrsgx_render.h"
+#include "pvrsgx_vm.h"
+
+/* Registers safe to read while the clocks are on. */
+static void dump_regs(struct seq_buf *sb, struct pvrsgx_device *sdev)
+{
+	const struct pvrsgx_core_regs *cr = sdev->cr;
+	u32 edm = pvrsgx_dirlist_edm(sdev), lists = sdev->family->num_dir_lists;
+	const struct {
+		const char *name;
+		u32 reg;
+	} regs[] = {
+		{ "CLKGATECTL", SGX_CR_CLKGATECTL },
+		{ "CLKGATECTL2", cr->clkgatectl2 },
+		{ "CLKGATESTATUS", cr->clkgatestatus },
+		{ "CORE_ID", cr->core_id },
+		{ "CORE_REVISION", cr->core_revision },
+		{ "SOFT_RESET", SGX_CR_SOFT_RESET },
+		{ "EVENT_STATUS", SGX_CR_EVENT_STATUS },
+		{ "EVENT_STATUS2", SGX_CR_EVENT_STATUS2 },
+		{ "EVENT_HOST_ENABLE", SGX_CR_EVENT_HOST_ENABLE },
+		{ "EVENT_HOST_ENABLE2", SGX_CR_EVENT_HOST_ENABLE2 },
+		{ "BIF_CTRL", SGX_CR_BIF_CTRL },
+		{ "BIF_INT_STAT", SGX_CR_BIF_INT_STAT },
+		{ "BIF_FAULT", SGX_CR_BIF_FAULT },
+		{ "BIF_BANK0", cr->bif_bank0 },
+		{ "BIF_DIR_LIST_BASE0", SGX_CR_BIF_DIR_LIST_BASE0 },
+		{ "BIF_DIR_LIST_BASE1", lists > 1 ? SGX_CR_BIF_DIR_LIST_BASE1 : 0 },
+		{ "BIF_DIR_LIST_BASE2", lists > 2 ? SGX_CR_BIF_DIR_LIST_BASE1 + 4 : 0 },
+		{ NULL, edm ? sgx_cr_bif_dir_list_base(edm) : 0 },
+		{ "EDM_TASK_REG0", cr->edm_task_reg0 },
+		{ "EDM_TASK_REG1", cr->edm_task_reg1 },
+	};
+
+	for (int i = 0; i < ARRAY_SIZE(regs); i++) {
+		char name[24];
+
+		/* 0 is a register this core lacks; only CLKGATECTL, the first, sits there. */
+		if (i && !regs[i].reg)
+			continue;
+		if (regs[i].name)
+			strscpy(name, regs[i].name);
+		else
+			snprintf(name, sizeof(name), "BIF_DIR_LIST_BASE%u", edm);
+		seq_buf_printf(sb, "%-20s 0x%04x 0x%08x\n", name, regs[i].reg,
+			       sgx_rd(sdev, regs[i].reg));
+	}
+}
+
+void pvrsgx_dump_status(struct seq_buf *sb, struct pvrsgx_device *sdev, bool regs)
+{
+	u32 *ctl = sdev->fw.cpu[PVRSGX_FW_CCBCTL];
+	struct pvrsgx_kobj_counts kc;
+
+	seq_buf_printf(sb, "firmware running: %d lost: %d resets: %u irqs: %d\n", sdev->fw_running,
+		       sdev->lost, sdev->reset_count, atomic_read(&sdev->irq_count));
+	seq_buf_printf(sb, "kernel PD 0x%08x idle PD 0x%08x\n",
+		       pvrsgx_mmu_pd_addr(sdev->kernel_mmu),
+		       pvrsgx_mmu_pd_addr(pvrsgx_idle_mmu(sdev)));
+	seq_buf_printf(sb, "ccb wo %u ro %u issued %llu consumed %llu\n", READ_ONCE(ctl[0]),
+		       READ_ONCE(ctl[1]), sdev->ccb_issued, pvrsgx_ccb_consumed(sdev));
+	seq_buf_printf(sb, "hostctl init 0x%x power 0x%x uk-lockups %u ",
+		       pvrsgx_fw_hostctl(sdev, HOSTCTL_INIT_STATUS),
+		       pvrsgx_fw_hostctl(sdev, HOSTCTL_POWER_STATUS),
+		       pvrsgx_fw_hostctl(sdev, HOSTCTL_UK_LOCKUPS));
+	seq_buf_printf(sb, "int 0x%x/0x%x assert 0x%x\n",
+		       pvrsgx_fw_hostctl(sdev, HOSTCTL_INTERRUPT_FLAGS),
+		       pvrsgx_fw_hostctl(sdev, HOSTCTL_INTERRUPT_CLEAR),
+		       pvrsgx_fw_hostctl(sdev, HOSTCTL_ASSERT_FAIL));
+	seq_buf_printf(sb, "powered: %d\n", READ_ONCE(sdev->powered));
+	if (pvrsgx_kobj_pending(sdev, &kc))
+		seq_buf_printf(sb, "kobjs graveyard %u (%llu pages) unmapped %u (%llu pages)\n",
+			       kc.graveyard, kc.graveyard_pages, kc.unmapped, kc.unmapped_pages);
+	seq_buf_printf(sb, "client unmaps waiting for the invalidate %u\n",
+		       pvrsgx_ccb_inval_pending(sdev));
+	/* After a failed start the clocks are still on; after a stop they are not. */
+	if (!regs || !READ_ONCE(sdev->powered))
+		return;
+	dump_regs(sb, sdev);
+}
+
+#define COREDUMP_BYTES	SZ_128K
+
+static void dump_lookup(struct seq_buf *sb, const char *name, struct pvrsgx_mmu *mmu, u32 addr)
+{
+	u32 pde, pte;
+
+	if (!mmu)
+		return;
+	if (pvrsgx_mmu_lookup(mmu, addr, &pde, &pte))
+		seq_buf_printf(sb, "  %s directory 0x%08x: pde 0x%08x pte 0x%08x\n", name,
+			       pvrsgx_mmu_pd_addr(mmu), pde, pte);
+	else
+		seq_buf_printf(sb, "  %s directory 0x%08x: busy\n", name, pvrsgx_mmu_pd_addr(mmu));
+}
+
+/*
+ * What the BIF was set to, and how each directory the job could have used
+ * translates the faulting address. After a start, so every register is safe
+ * to read.
+ */
+static void dump_bif(struct seq_buf *sb, struct pvrsgx_device *sdev, struct pvrsgx_job *bad)
+{
+	const struct pvrsgx_core_regs *cr = sdev->cr;
+	u32 stat = sgx_rd(sdev, SGX_CR_BIF_INT_STAT);
+	u32 addr = sgx_rd(sdev, SGX_CR_BIF_FAULT) & cr->bif_fault_addr;
+
+	if (cr->bif_bank0)
+		seq_buf_printf(sb, "BIF: bank set 0x%08x bank0 0x%08x mem req 0x%08x\n",
+			       sgx_rd(sdev, cr->bif_bank_set), sgx_rd(sdev, cr->bif_bank0),
+			       sgx_rd(sdev, SGX_CR_BIF_MEM_REQ_STAT));
+	else
+		seq_buf_printf(sb, "BIF: mem req 0x%08x\n", sgx_rd(sdev, SGX_CR_BIF_MEM_REQ_STAT));
+	for (unsigned int i = 0; i < sdev->family->num_dir_lists; i++)
+		seq_buf_printf(sb, "  dir list %2u: 0x%08x\n", i,
+			       sgx_rd(sdev, sgx_cr_bif_dir_list_base(i)));
+	for (unsigned int i = 0; i < 16; i++)
+		seq_buf_printf(sb, "  use code base %2u: 0x%08x\n", i,
+			       sgx_rd(sdev, cr->use_code_base0 + 4 * i));
+	seq_buf_printf(sb, "  pds exec base: 0x%08x\n", sgx_rd(sdev, cr->pds_exec_base));
+	if (!(stat & cr->bif_int_stat_fault_req))
+		return;
+	seq_buf_printf(sb, "fault at 0x%08x: requestor 0x%04x type %u\n", addr,
+		       field_get(cr->bif_int_stat_fault_req, stat),
+		       field_get(cr->bif_int_stat_fault_type, stat));
+	dump_lookup(sb, "job's", bad->ctx->vm->mmu, addr);
+	dump_lookup(sb, "firmware's", sdev->kernel_mmu, addr);
+	if (!pvrsgx_core_single_context(sdev->family))
+		dump_lookup(sb, "idle", sdev->idle_mmu, addr);
+}
+
+/*
+ * Called from the timeout handler with the scheduler stopped, before the
+ * restart: what the microkernel and the hung job left behind.
+ */
+void pvrsgx_coredump(struct pvrsgx_device *sdev, struct pvrsgx_job *bad)
+{
+	static const char *const types[] = {
+		[DRM_POWERVR_SGX_JOB_TYPE_RENDER] = "render",
+		[DRM_POWERVR_SGX_JOB_TYPE_TRANSFER] = "transfer",
+		[DRM_POWERVR_SGX_JOB_TYPE_NULL] = "null",
+	};
+	unsigned long flags;
+	struct seq_buf sb;
+	char *buf;
+
+	/* The timeout handler signals fences: no reclaim, so a dump can be lost but never waits. */
+	buf = __vmalloc(COREDUMP_BYTES, GFP_NOWAIT | __GFP_NOWARN);
+	if (!buf)
+		return;
+	seq_buf_init(&sb, buf, COREDUMP_BYTES);
+	seq_buf_printf(&sb, "powervr_sgx hang: %s job, ccb %llu, reset %u\n",
+		       bad->type < ARRAY_SIZE(types) ? types[bad->type] : "?", bad->ccb_seq,
+		       sdev->reset_count + 1);
+	pvrsgx_dump_status(&sb, sdev, true);
+	if (sdev->fw_running)
+		dump_bif(&sb, sdev, bad);
+	pvrsgx_fw_dump(&sb, sdev, PVRSGX_FW_HOSTCTL, "hostctl");
+	pvrsgx_fw_dump(&sb, sdev, PVRSGX_FW_TA3DCTL, "ta3dctl");
+	pvrsgx_fw_dump(&sb, sdev, PVRSGX_FW_CCBCTL, "ccbctl");
+	pvrsgx_fw_dump(&sb, sdev, PVRSGX_FW_CCB, "ccb");
+	pvrsgx_job_dump(&sb, bad);
+	if (seq_buf_has_overflowed(&sb))
+		drm_warn(&sdev->base, "coredump truncated\n");
+	/* devcoredump registers a device, which allocates: from a worker. */
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	if (!sdev->coredump) {
+		sdev->coredump = buf;
+		sdev->coredump_len = seq_buf_used(&sb);
+		buf = NULL;
+	}
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	vfree(buf);
+	queue_work(system_dfl_wq, &sdev->coredump_work);
+}
+
+static void coredump_work_fn(struct work_struct *work)
+{
+	struct pvrsgx_device *sdev = container_of(work, struct pvrsgx_device, coredump_work);
+	unsigned long flags;
+	size_t len;
+	void *buf;
+
+	spin_lock_irqsave(&sdev->fence_lock, flags);
+	buf = sdev->coredump;
+	len = sdev->coredump_len;
+	sdev->coredump = NULL;
+	spin_unlock_irqrestore(&sdev->fence_lock, flags);
+	if (buf)
+		dev_coredumpv(sdev->base.dev, buf, len, GFP_KERNEL);
+}
+
+void pvrsgx_coredump_init(struct pvrsgx_device *sdev)
+{
+	INIT_WORK(&sdev->coredump_work, coredump_work_fn);
+}
+
+void pvrsgx_coredump_fini(struct pvrsgx_device *sdev)
+{
+	cancel_work_sync(&sdev->coredump_work);
+	vfree(sdev->coredump);
+	sdev->coredump = NULL;
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_dump.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_dump.h
new file mode 100644
index 000000000..f47db524f
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_dump.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_DUMP_H
+#define PVRSGX_DUMP_H
+
+struct pvrsgx_device;
+struct pvrsgx_job;
+struct seq_buf;
+
+void pvrsgx_dump_status(struct seq_buf *sb, struct pvrsgx_device *sdev, bool regs);
+void pvrsgx_coredump(struct pvrsgx_device *sdev, struct pvrsgx_job *bad);
+void pvrsgx_coredump_init(struct pvrsgx_device *sdev);
+void pvrsgx_coredump_fini(struct pvrsgx_device *sdev);
+
+#endif /* PVRSGX_DUMP_H */
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_of.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_of.c
new file mode 100644
index 000000000..e347a3699
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_of.c
@@ -0,0 +1,119 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * An SGX on a device tree SoC (Documentation/devicetree/bindings/gpu/
+ * img,powervr-sgx.yaml). On TI parts the GPU node has no clocks or power
+ * domain of its own: its ti-sysc parent module enables both, and applies the
+ * SGX interrupt quirk, when the GPU device is runtime-resumed. Every register
+ * access happens with the parent held awake.
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/io.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/property.h>
+
+#include "pvrsgx_device.h"
+#include "pvrsgx_drv.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_pm.h"
+
+static int pvrsgx_of_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct pvrsgx_device *sdev;
+	struct resource *res, win;
+	int err, irq;
+
+	/* Clocks and resets on the GPU node itself (non-TI parts) aren't handled yet. */
+	if (of_property_present(dev->of_node, "clocks") ||
+	    of_property_present(dev->of_node, "resets")) {
+		dev_err(dev, "GPU node with its own clocks or resets; not touching it\n");
+		return -ENODEV;
+	}
+	if (!dev->parent || !pm_runtime_enabled(dev->parent)) {
+		dev_err(dev, "no runtime-PM parent to clock the GPU; not touching it\n");
+		return -ENODEV;
+	}
+
+	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+	if (!res || resource_size(res) < PVRSGX_REGS_MIN)
+		return -ENODEV;
+	irq = platform_get_irq(pdev, 0);
+	if (irq < 0)
+		return irq;
+
+	err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+	if (err)
+		return err;
+	dma_set_max_seg_size(dev, UINT_MAX);
+
+	sdev = pvrsgx_alloc(dev);
+	if (IS_ERR(sdev))
+		return PTR_ERR(sdev);
+	sdev->family = pvrsgx_core_family_get((uintptr_t)device_get_match_data(dev));
+	sdev->irq = irq;
+	platform_set_drvdata(pdev, sdev);
+
+	/* Map only the registers: nothing past them is named (OMAP3 maps 32 MiB). */
+	win = DEFINE_RES_MEM_NAMED(res->start, PVRSGX_REGS_MIN, PVRSGX_DRIVER_NAME);
+	sdev->regs = devm_ioremap_resource(dev, &win);
+	if (IS_ERR(sdev->regs))
+		return PTR_ERR(sdev->regs);
+
+	/* The module is clocked only while its parent is awake. */
+	err = pm_runtime_resume_and_get(dev->parent);
+	if (err)
+		return err;
+	err = pvrsgx_probe_common(sdev);
+	pm_runtime_put(dev->parent);
+	return err;
+}
+
+static void pvrsgx_of_remove(struct platform_device *pdev)
+{
+	pvrsgx_remove_common(platform_get_drvdata(pdev));
+}
+
+static void pvrsgx_of_shutdown(struct platform_device *pdev)
+{
+	struct pvrsgx_device *sdev = platform_get_drvdata(pdev);
+
+	/* Only a powered core can be stopped. */
+	if (sdev && pm_runtime_get_if_active(&pdev->dev) > 0) {
+		pvrsgx_fw_stop(sdev);
+		pm_runtime_put_noidle(&pdev->dev);
+	}
+}
+
+static const struct of_device_id pvrsgx_of_ids[] = {
+	{ .compatible = "img,powervr-sgx530", .data = (void *)530 },
+	{ }
+};
+MODULE_DEVICE_TABLE(of, pvrsgx_of_ids);
+
+static struct platform_driver pvrsgx_of_driver = {
+	.probe = pvrsgx_of_probe,
+	.remove = pvrsgx_of_remove,
+	.shutdown = pvrsgx_of_shutdown,
+	.driver = {
+		.name = PVRSGX_DRIVER_NAME,
+		.of_match_table = pvrsgx_of_ids,
+		.pm = pm_ptr(&pvrsgx_pm_ops),
+	},
+};
+
+int pvrsgx_of_register(void)
+{
+	return platform_driver_register(&pvrsgx_of_driver);
+}
+
+void pvrsgx_of_unregister(void)
+{
+	platform_driver_unregister(&pvrsgx_of_driver);
+}
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_pm.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_pm.c
new file mode 100644
index 000000000..24c93a91b
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_pm.c
@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+#include <drm/drm_print.h>
+
+#include <linux/pm_runtime.h>
+
+#include "pvrsgx_device.h"
+#include "pvrsgx_fw.h"
+#include "pvrsgx_job.h"
+#include "pvrsgx_pm.h"
+
+/*
+ * Runtime PM. The SGX function has no PCI PM capability, so the PCI core keeps
+ * it in D0: suspend is the microkernel's power-off (pvrsgx_fw_stop), resume
+ * the same full start a reset does. A job on the GPU holds a reference from
+ * run_job until it leaves the active list.
+ */
+#define AUTOSUSPEND_MS	500
+
+int pvrsgx_power_get(struct pvrsgx_device *sdev)
+{
+	return pm_runtime_resume_and_get(sdev->base.dev);
+}
+
+void pvrsgx_power_put(struct pvrsgx_device *sdev, u32 n)
+{
+	if (!n)
+		return;
+	pm_runtime_mark_last_busy(sdev->base.dev);
+	while (n--)
+		pm_runtime_put_autosuspend(sdev->base.dev);
+}
+
+/*
+ * After probe. The PCI core holds a reference over probe and forbids runtime
+ * PM by default; an auxiliary device starts with runtime PM disabled, and its
+ * parent (gma500, which powers the graphics island) stays up while it runs.
+ */
+void pvrsgx_power_init(struct pvrsgx_device *sdev)
+{
+	struct device *dev = sdev->base.dev;
+
+	pm_runtime_set_autosuspend_delay(dev, AUTOSUSPEND_MS);
+	pm_runtime_use_autosuspend(dev);
+	pm_runtime_mark_last_busy(dev);
+	if (sdev->pdev) {
+		pm_runtime_put_autosuspend(dev);
+		pm_runtime_allow(dev);
+	} else {
+		pm_runtime_set_active(dev);
+		pm_runtime_enable(dev);
+		/*
+		 * A single-context core is still in bring-up: its power-off and
+		 * the state loss of a suspend are untested, so it stays awake.
+		 */
+		if (pvrsgx_core_single_context(sdev->family)) {
+			pm_runtime_get_noresume(dev);
+			sdev->pm_pinned = true;
+		}
+	}
+}
+
+/* Before remove: undo pvrsgx_power_init, with the device awake. */
+void pvrsgx_power_fini(struct pvrsgx_device *sdev)
+{
+	struct device *dev = sdev->base.dev;
+
+	if (sdev->pdev) {
+		pm_runtime_forbid(dev);
+		pm_runtime_dont_use_autosuspend(dev);
+		pm_runtime_get_noresume(dev);
+	} else {
+		pm_runtime_get_sync(dev);
+		pm_runtime_dont_use_autosuspend(dev);
+		pm_runtime_disable(dev);
+		pm_runtime_set_suspended(dev);
+		pm_runtime_put_noidle(dev);
+		if (sdev->pm_pinned) {
+			pm_runtime_put_noidle(dev);
+			sdev->pm_pinned = false;
+		}
+	}
+}
+
+static int pvrsgx_runtime_suspend(struct device *dev)
+{
+	struct pvrsgx_device *sdev = dev_get_drvdata(dev);
+
+	if (!sdev->lost)
+		pvrsgx_fw_stop(sdev);
+	return 0;
+}
+
+static int pvrsgx_runtime_resume(struct device *dev)
+{
+	struct pvrsgx_device *sdev = dev_get_drvdata(dev);
+
+	/* A failed start loses the device: run_job then fails every job with -EIO. */
+	if (!sdev->lost && pvrsgx_fw_start(sdev)) {
+		drm_err(&sdev->base, "microkernel did not start on resume: device lost\n");
+		sdev->lost = true;
+	}
+	return 0;
+}
+
+/*
+ * System sleep. The core loses power, so the microkernel has to stop first and
+ * start again after: drain the GPU, then the runtime suspend and resume above.
+ * Behind gma500 this runs before its suspend and after its resume, with the
+ * graphics island up.
+ */
+static int pvrsgx_suspend(struct device *dev)
+{
+	struct pvrsgx_device *sdev = dev_get_drvdata(dev);
+	int err = pvrsgx_job_quiesce(sdev);
+
+	if (err)
+		return err;
+	err = pm_runtime_force_suspend(dev);
+	if (err)
+		pvrsgx_job_unquiesce(sdev);
+	return err;
+}
+
+static int pvrsgx_resume(struct device *dev)
+{
+	struct pvrsgx_device *sdev = dev_get_drvdata(dev);
+	int err = pm_runtime_force_resume(dev);
+
+	pvrsgx_job_unquiesce(sdev);
+	return err;
+}
+
+const struct dev_pm_ops pvrsgx_pm_ops = {
+	SYSTEM_SLEEP_PM_OPS(pvrsgx_suspend, pvrsgx_resume)
+	RUNTIME_PM_OPS(pvrsgx_runtime_suspend, pvrsgx_runtime_resume, NULL)
+};
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_pm.h b/drivers/gpu/drm/powervr_sgx/pvrsgx_pm.h
new file mode 100644
index 000000000..62d6ebaa1
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_pm.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+#ifndef PVRSGX_PM_H
+#define PVRSGX_PM_H
+
+#include <linux/kconfig.h>
+#include <linux/pm.h>
+#include <linux/types.h>
+
+struct pvrsgx_device;
+
+/* Referenced through pm_ptr(), so declared either way. */
+extern const struct dev_pm_ops pvrsgx_pm_ops;
+
+#if IS_ENABLED(CONFIG_PM)
+
+int pvrsgx_power_get(struct pvrsgx_device *sdev);
+void pvrsgx_power_put(struct pvrsgx_device *sdev, u32 n);
+void pvrsgx_power_init(struct pvrsgx_device *sdev);
+void pvrsgx_power_fini(struct pvrsgx_device *sdev);
+#else
+/* Without runtime PM the GPU stays on from probe to remove. */
+static inline int pvrsgx_power_get(struct pvrsgx_device *sdev) { return 0; }
+static inline void pvrsgx_power_put(struct pvrsgx_device *sdev, u32 n) { }
+static inline void pvrsgx_power_init(struct pvrsgx_device *sdev) { }
+static inline void pvrsgx_power_fini(struct pvrsgx_device *sdev) { }
+#endif
+
+#endif /* PVRSGX_PM_H */

-- 
2.50.1

[PATCH RFC 13/13] drm/powervr_sgx: Add KUnit tests

[Joseph Telaak]

Tests that need no GPU:

- powervr_sgx_fw: a well-formed firmware file is accepted, and each
  malformed header, table and checksum is refused.
- powervr_sgx_kobj: the chain of kernel objects a released client
  leaves behind is freed at unbind.
- powervr_sgx_core: the core table's lookups and entries.
- powervr_sgx_equiv: every register write and every structure the
  kernel builds for the microkernel on the SGX545, through a start, a
  command, a recovery, a stop, a resume, contexts and a two-frame
  render, checked against a recorded golden.
- powervr_sgx_ccb, powervr_sgx_rt, powervr_sgx_usse,
  powervr_sgx_validate and powervr_sgx_prime: the command ring,
  render-target tiling numbers, the instruction table, the shader
  validator and a job's cache ranges.

Run them with:

  ./tools/testing/kunit/kunit.py run --arch=x86_64 \
      --kunitconfig=drivers/gpu/drm/powervr_sgx/tests/.kunitconfig

The tests are included into the driver's own objects, and with them built
in, the register write helper calls a test hook so the equivalence test
can record every write.

Signed-off-by: Joseph Telaak <joseph@telaak.dev>
---
 drivers/gpu/drm/powervr_sgx/Kconfig                |  16 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c           |   4 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_core.c          |   4 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c            |   5 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c         |   4 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_render.c        |   5 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c          |   4 +
 drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c      |   4 +
 drivers/gpu/drm/powervr_sgx/tests/.kunitconfig     |   5 +
 .../gpu/drm/powervr_sgx/tests/pvrsgx_ccb_test.c    | 111 +++++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_core_test.c   | 119 +++++
 .../drm/powervr_sgx/tests/pvrsgx_equiv_golden.h    |  32 ++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_equiv_test.c  | 555 +++++++++++++++++++++
 drivers/gpu/drm/powervr_sgx/tests/pvrsgx_fw_test.c | 168 +++++++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_kobj_test.c   | 203 ++++++++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_prime_test.c  | 125 +++++
 drivers/gpu/drm/powervr_sgx/tests/pvrsgx_rt_test.c |  63 +++
 .../gpu/drm/powervr_sgx/tests/pvrsgx_usse_test.c   | 145 ++++++
 .../drm/powervr_sgx/tests/pvrsgx_validate_test.c   |  32 ++
 19 files changed, 1604 insertions(+)

diff --git a/drivers/gpu/drm/powervr_sgx/Kconfig b/drivers/gpu/drm/powervr_sgx/Kconfig
index dbbba1d68..6c880ffef 100644
--- a/drivers/gpu/drm/powervr_sgx/Kconfig
+++ b/drivers/gpu/drm/powervr_sgx/Kconfig
@@ -47,3 +47,19 @@ config DRM_POWERVR_SGX_OF
 	  bring-up. Dma-buf import isn't supported on ARM yet.
 
 	  Say N unless this kernel is for such an SoC.
+
+config DRM_POWERVR_SGX_KUNIT_TEST
+	tristate "KUnit tests for the PowerVR SGX5xx driver" if !KUNIT_ALL_TESTS
+	depends on DRM_POWERVR_SGX && KUNIT
+	select DRM_KUNIT_TEST_HELPERS
+	default KUNIT_ALL_TESTS
+	help
+	  Unit tests of the driver's firmware file checks (a well-formed
+	  image is accepted, and every malformed header, table and checksum
+	  is refused), of kernel-object teardown at unbind, of the core
+	  table, of a job's cache ranges and of a render target's tiling
+	  numbers, and a record of the SGX545's register writes and
+	  firmware structures that a change must leave as it is. The tests
+	  need no GPU.
+
+	  For developers; if unsure, say N.
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c
index 351a7f930..1698ff9f0 100644
--- a/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_ccb.c
@@ -202,3 +202,7 @@ void pvrsgx_ccb_inval_reap(struct pvrsgx_device *sdev, bool all)
 		w->done(w);
 	}
 }
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+#include "tests/pvrsgx_ccb_test.c"
+#endif
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_core.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_core.c
index dd8fd14eb..ebc0cba0b 100644
--- a/drivers/gpu/drm/powervr_sgx/pvrsgx_core.c
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_core.c
@@ -294,3 +294,7 @@ const struct pvrsgx_core_info *pvrsgx_core_lookup(const struct pvrsgx_core_famil
 			return &cores[i];
 	return NULL;
 }
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+#include "tests/pvrsgx_core_test.c"
+#endif
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c
index b3c0d1891..4787ed69a 100644
--- a/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_fw.c
@@ -729,3 +729,8 @@ void pvrsgx_fw_stop(struct pvrsgx_device *sdev)
 		sgx_rd(sdev, SGX_CR_SOFT_RESET);
 	}
 }
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+#include "tests/pvrsgx_fw_test.c"
+#include "tests/pvrsgx_equiv_test.c"
+#endif
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c
index e6aca1145..cb5ddf212 100644
--- a/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_prime.c
@@ -201,3 +201,7 @@ void pvrsgx_prime_put(struct pvrsgx_job *job)
 		drm_gem_object_put(job->ranges[i].obj);
 	kvfree(job->ranges);
 }
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+#include "tests/pvrsgx_prime_test.c"
+#endif
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_render.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_render.c
index 00b444d05..67e7dfc11 100644
--- a/drivers/gpu/drm/powervr_sgx/pvrsgx_render.c
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_render.c
@@ -950,3 +950,8 @@ void pvrsgx_job_dump(struct seq_buf *sb, struct pvrsgx_job *job)
 	else if (job->type == DRM_POWERVR_SGX_JOB_TYPE_TRANSFER)
 		pvrsgx_transfer_dump(sb, job);
 }
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+#include "tests/pvrsgx_kobj_test.c"
+#include "tests/pvrsgx_rt_test.c"
+#endif
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c
index 9aa0cd4ab..38d2bd302 100644
--- a/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_usse.c
@@ -234,3 +234,7 @@ const char *pvrsgx_usse_refusal_name(enum pvrsgx_usse_refusal why)
 
 	return why < ARRAY_SIZE(names) ? names[why] : "?";
 }
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+#include "tests/pvrsgx_usse_test.c"
+#endif
diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c
index cf9cfcf87..4a52ff10d 100644
--- a/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c
+++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_validate.c
@@ -186,3 +186,7 @@ out_obj:
 	drm_gem_object_put(obj);
 	return err;
 }
+
+#if IS_ENABLED(CONFIG_DRM_POWERVR_SGX_KUNIT_TEST)
+#include "tests/pvrsgx_validate_test.c"
+#endif
diff --git a/drivers/gpu/drm/powervr_sgx/tests/.kunitconfig b/drivers/gpu/drm/powervr_sgx/tests/.kunitconfig
new file mode 100644
index 000000000..3bf1e8f2c
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/.kunitconfig
@@ -0,0 +1,5 @@
+CONFIG_KUNIT=y
+CONFIG_PCI=y
+CONFIG_DRM=y
+CONFIG_DRM_POWERVR_SGX=y
+CONFIG_DRM_POWERVR_SGX_KUNIT_TEST=y
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_ccb_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_ccb_test.c
new file mode 100644
index 000000000..fee8a7fe8
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_ccb_test.c
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * KUnit tests of the kernel CCB's cache control, included from
+ * pvrsgx_ccb.c. The ring, its control words and the registers are memory;
+ * no microkernel takes commands, so the read offset never moves.
+ */
+
+#include <kunit/test.h>
+
+#include "../pvrsgx_drv.h"
+
+/* A device on the SGX545's table, with quirks replaced. */
+static struct pvrsgx_device *ccb_test_device(struct kunit *test, u32 quirks)
+{
+	struct pvrsgx_device *sdev = kunit_kzalloc(test, sizeof(*sdev), GFP_KERNEL);
+	struct pvrsgx_core_family *family = kunit_kzalloc(test, sizeof(*family), GFP_KERNEL);
+	void *regs = kunit_kzalloc(test, PVRSGX_REGS_MIN, GFP_KERNEL);
+
+	KUNIT_ASSERT_NOT_NULL(test, sdev);
+	KUNIT_ASSERT_NOT_NULL(test, family);
+	KUNIT_ASSERT_NOT_NULL(test, regs);
+	KUNIT_ASSERT_NOT_NULL(test, pvrsgx_core_family_get(545));
+	*family = *pvrsgx_core_family_get(545);
+	family->quirks = quirks;
+	sdev->family = family;
+	sdev->cr = family->regs;
+	sdev->regs = (void __iomem __force *)regs;
+	sdev->fw.cpu[PVRSGX_FW_CCB] = kunit_kzalloc(test, PVRSGX_CCB_SLOTS *
+						    PVRSGX_CCB_CMD_WORDS * 4, GFP_KERNEL);
+	sdev->fw.cpu[PVRSGX_FW_CCBCTL] = kunit_kzalloc(test, 8, GFP_KERNEL);
+	sdev->fw.cpu[PVRSGX_FW_KICKER] = kunit_kzalloc(test, 4, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_NULL(test, sdev->fw.cpu[PVRSGX_FW_CCB]);
+	KUNIT_ASSERT_NOT_NULL(test, sdev->fw.cpu[PVRSGX_FW_CCBCTL]);
+	KUNIT_ASSERT_NOT_NULL(test, sdev->fw.cpu[PVRSGX_FW_KICKER]);
+	mutex_init(&sdev->ccb_lock);
+	spin_lock_init(&sdev->ccb_spin);
+	return sdev;
+}
+
+static const u32 *ccb_test_cmd(struct pvrsgx_device *sdev, unsigned int slot)
+{
+	return (u32 *)sdev->fw.cpu[PVRSGX_FW_CCB] + slot * PVRSGX_CCB_CMD_WORDS;
+}
+
+/* Without the quirk (the SGX545) a BIF and a data invalidate go together, as before. */
+static void ccb_test_inval_together(struct kunit *test)
+{
+	struct pvrsgx_device *sdev = ccb_test_device(test, 0);
+	static const u32 data[6] = { 1, 2, 3, 4, 5, 6 };
+	u64 seq;
+
+	sdev->fw.cache_control = PVRSGX_CC_INVAL_BIF_PD | PVRSGX_CC_INVAL_DATA;
+	KUNIT_ASSERT_EQ(test, pvrsgx_ccb_send(sdev, PVRSGX_KCMD_TA, data, &seq), 0);
+	KUNIT_EXPECT_EQ(test, seq, 1);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 0)[0], PVRSGX_KCMD_TA);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 0)[1],
+			PVRSGX_CC_INVAL_BIF_PD | PVRSGX_CC_INVAL_DATA);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 0)[2], 1);
+	KUNIT_EXPECT_EQ(test, sdev->fw.cache_control, 0);
+}
+
+/*
+ * BRN_28889: the BIF part goes alone in a PROCESS_QUEUES, and the command
+ * with the data part waits until the microkernel has taken it. Nothing takes
+ * it here, so the send gives up, bounded, with the job's command unwritten
+ * and the invalidates kept for the next try.
+ */
+static void ccb_test_inval_split(struct kunit *test)
+{
+	struct pvrsgx_device *sdev = ccb_test_device(test, PVRSGX_QUIRK_BRN_28889);
+	static const u32 data[6] = { 1, 2, 3, 4, 5, 6 };
+	u64 seq = 0;
+
+	sdev->fw.cache_control = PVRSGX_CC_INVAL_BIF_PD | PVRSGX_CC_INVAL_DATA;
+	KUNIT_EXPECT_EQ(test, pvrsgx_ccb_send(sdev, PVRSGX_KCMD_TA, data, &seq), -ETIMEDOUT);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 0)[0], PVRSGX_KCMD_PROCESS_QUEUES);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 0)[1], PVRSGX_CC_INVAL_BIF_PD);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 0)[2], 0);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 1)[0], 0);
+	KUNIT_EXPECT_EQ(test, seq, 0);
+	KUNIT_EXPECT_EQ(test, sdev->fw.cache_control,
+			PVRSGX_CC_INVAL_BIF_PD | PVRSGX_CC_INVAL_DATA);
+}
+
+/* One kind of invalidate alone needs no split, with the quirk or without. */
+static void ccb_test_inval_alone(struct kunit *test)
+{
+	struct pvrsgx_device *sdev = ccb_test_device(test, PVRSGX_QUIRK_BRN_28889);
+	static const u32 data[6];
+
+	sdev->fw.cache_control = PVRSGX_CC_INVAL_DATA;
+	KUNIT_ASSERT_EQ(test, pvrsgx_ccb_send(sdev, PVRSGX_KCMD_TA, data, NULL), 0);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 0)[0], PVRSGX_KCMD_TA);
+	KUNIT_EXPECT_EQ(test, ccb_test_cmd(sdev, 0)[1], PVRSGX_CC_INVAL_DATA);
+}
+
+static struct kunit_case pvrsgx_ccb_test_cases[] = {
+	KUNIT_CASE(ccb_test_inval_together),
+	KUNIT_CASE(ccb_test_inval_split),
+	KUNIT_CASE(ccb_test_inval_alone),
+	{}
+};
+
+static struct kunit_suite pvrsgx_ccb_test_suite = {
+	.name = "powervr_sgx_ccb",
+	.test_cases = pvrsgx_ccb_test_cases,
+};
+
+kunit_test_suite(pvrsgx_ccb_test_suite);
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_core_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_core_test.c
new file mode 100644
index 000000000..ebc7059ed
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_core_test.c
@@ -0,0 +1,119 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/* KUnit tests of the core table, included from pvrsgx_core.c. */
+
+#include <kunit/test.h>
+
+static void core_test_lookup(struct kunit *test)
+{
+	const struct pvrsgx_core_family *f = pvrsgx_core_family_get(545);
+
+	KUNIT_ASSERT_NOT_NULL(test, f);
+	KUNIT_EXPECT_STREQ(test, pvrsgx_core_lookup(f, 0x01150000, 0x0001000e)->name,
+			   "Intel CE5300 (SGX545 1.0.14)");
+	KUNIT_EXPECT_STREQ(test, pvrsgx_core_lookup(f, 0x01150000, 0x0001000d)->name,
+			   "Intel Cedarview (SGX545 1.0.13)");
+	KUNIT_EXPECT_NULL(test, pvrsgx_core_lookup(f, 0x01150000, 0x0001000c));
+	KUNIT_EXPECT_NULL(test, pvrsgx_core_lookup(f, 0x01120000, 0x0001000e));
+}
+
+static void core_test_unknown(struct kunit *test)
+{
+	KUNIT_EXPECT_NULL(test, pvrsgx_core_family_get(0));
+	KUNIT_EXPECT_NULL(test, pvrsgx_core_family_get(535));
+	KUNIT_EXPECT_NULL(test, pvrsgx_core_family_get(544));
+}
+
+/* Every entry is one the driver can drive, with the registers it reads first. */
+static void core_test_table(struct kunit *test)
+{
+	for (unsigned int i = 0; i < ARRAY_SIZE(families); i++) {
+		const struct pvrsgx_core_family *f = &families[i];
+
+		KUNIT_EXPECT_TRUE(test, pvrsgx_core_family_usable(f));
+		KUNIT_EXPECT_NE(test, f->regs->core_id, 0);
+		KUNIT_EXPECT_NE(test, f->regs->core_revision, 0);
+		KUNIT_EXPECT_EQ(test, !!(f->features & PVRSGX_FEATURE_MULTI_EVENT_KICK),
+				!!f->regs->event_kick2);
+		KUNIT_EXPECT_NE(test, f->regs->use_code_base0, 0);
+		KUNIT_EXPECT_GT(test, f->num_init_regs, 0);
+	}
+	for (unsigned int i = 0; i < ARRAY_SIZE(cores); i++) {
+		KUNIT_EXPECT_NOT_NULL(test, cores[i].family);
+		KUNIT_EXPECT_NOT_NULL(test, cores[i].fw_name);
+		KUNIT_EXPECT_GT(test, cores[i].clock_hz, 0);
+	}
+}
+
+/*
+ * The SGX545's table holds what the driver used before the table had it:
+ * the heap constants, the shifts and the registers.
+ */
+static void core_test_sgx545_values(struct kunit *test)
+{
+	const struct pvrsgx_core_family *f = pvrsgx_core_family_get(545);
+	const struct pvrsgx_heap_layout *h;
+
+	KUNIT_ASSERT_NOT_NULL(test, f);
+	h = f->heaps;
+	KUNIT_ASSERT_NOT_NULL(test, h);
+	KUNIT_EXPECT_EQ(test, h->params_3d.base, PVRSGX_HEAP_3D_PARAMS);
+	KUNIT_EXPECT_EQ(test, h->ta_data.base, PVRSGX_HEAP_TA_DATA);
+	KUNIT_EXPECT_EQ(test, h->sync.base, PVRSGX_HEAP_SYNC);
+	KUNIT_EXPECT_EQ(test, h->pds_pixel.base, PVRSGX_HEAP_PDS_PIXEL);
+	KUNIT_EXPECT_EQ(test, h->kernel_code.base, PVRSGX_HEAP_KERNEL_CODE);
+	KUNIT_EXPECT_EQ(test, h->pds_vertex.base, PVRSGX_HEAP_PDS_VERTEX);
+	KUNIT_EXPECT_EQ(test, h->kernel_data.base, PVRSGX_HEAP_KERNEL_DATA);
+	KUNIT_EXPECT_EQ(test, h->usse_pixel.base, PVRSGX_HEAP_USSE_PIXEL);
+	KUNIT_EXPECT_EQ(test, h->usse_pixel.end - h->usse_pixel.base, PVRSGX_HEAP_USSE_PIXEL_SIZE);
+	KUNIT_EXPECT_EQ(test, h->usse_vertex.base, PVRSGX_HEAP_USSE_VERTEX);
+	KUNIT_EXPECT_EQ(test, h->usse_vertex.end - h->usse_vertex.base,
+			PVRSGX_HEAP_USSE_VERTEX_SIZE);
+	KUNIT_EXPECT_EQ(test, h->general.base, PVRSGX_HEAP_GENERAL);
+	KUNIT_EXPECT_EQ(test, h->texture.base, PVRSGX_HEAP_TEXTURE);
+	KUNIT_EXPECT_EQ(test, h->texture.end, SZ_4G);
+	KUNIT_EXPECT_EQ(test, h->pb_max_pages, 0x8000);
+	KUNIT_EXPECT_EQ(test, f->quirks, 0);
+	KUNIT_EXPECT_EQ(test, f->regs->bif_fault_addr, SGX_MMU_ADDR_MASK);
+	KUNIT_EXPECT_EQ(test, f->regs->bif_bank_set, SGX_CR_BIF_BANK_SET);
+	KUNIT_EXPECT_EQ(test, f->regs->bif_bank0, SGX_CR_BIF_BANK0);
+	KUNIT_EXPECT_EQ(test, f->regs->use_code_base_addr_shift, 7);
+	KUNIT_EXPECT_EQ(test, f->regs->use_code_base_dm_shift, 25);
+	KUNIT_EXPECT_EQ(test, f->regs->cr_b30, 0x0b30);
+}
+
+/* The SGX530: one directory list, 28 bits, BRN_28889, a layout the EDM reaches. */
+static void core_test_sgx530(struct kunit *test)
+{
+	const struct pvrsgx_core_family *f = pvrsgx_core_family_get(530);
+
+	KUNIT_ASSERT_NOT_NULL(test, f);
+	KUNIT_EXPECT_TRUE(test, pvrsgx_core_single_context(f));
+	KUNIT_EXPECT_EQ(test, f->num_dir_lists, 1);
+	KUNIT_EXPECT_EQ(test, f->va_bits, 28);
+	KUNIT_EXPECT_EQ(test, f->quirks, PVRSGX_QUIRK_BRN_28889);
+	KUNIT_EXPECT_EQ(test, f->regs->event_kick2, 0);
+	KUNIT_EXPECT_EQ(test, f->regs->bif_bank0, 0);
+	KUNIT_EXPECT_EQ(test, f->regs->clkgatectl2, 0);
+	KUNIT_EXPECT_TRUE(test, pvrsgx_core_family_usable(f));
+	KUNIT_EXPECT_FALSE(test, pvrsgx_core_single_context(pvrsgx_core_family_get(545)));
+	KUNIT_EXPECT_STREQ(test, pvrsgx_core_lookup(f, 0x01120000, 0x00010205)->name,
+			   "TI OMAP3630 (SGX530 1.2.5)");
+}
+
+static struct kunit_case pvrsgx_core_test_cases[] = {
+	KUNIT_CASE(core_test_lookup),
+	KUNIT_CASE(core_test_unknown),
+	KUNIT_CASE(core_test_table),
+	KUNIT_CASE(core_test_sgx545_values),
+	KUNIT_CASE(core_test_sgx530),
+	{}
+};
+
+static struct kunit_suite pvrsgx_core_test_suite = {
+	.name = "powervr_sgx_core",
+	.test_cases = pvrsgx_core_test_cases,
+};
+
+kunit_test_suite(pvrsgx_core_test_suite);
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_equiv_golden.h b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_equiv_golden.h
new file mode 100644
index 000000000..635481da7
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_equiv_golden.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The SGX545 records of pvrsgx_equiv_test.c: each phase's line count and
+ * CRC32. Change these only for a change meant to change the SGX545's
+ * behavior, and say so in the commit message.
+ */
+
+#ifndef PVRSGX_EQUIV_GOLDEN_H
+#define PVRSGX_EQUIV_GOLDEN_H
+
+struct pvrsgx_equiv_golden {
+	const char *phase;
+	u32 lines;
+	u32 crc;
+};
+
+static const struct pvrsgx_equiv_golden pvrsgx_equiv_golden_545[] = {
+	{ "init", 102, 0x0df03815 },
+	{ "start", 82, 0xb260b7dd },
+	{ "send", 20, 0x09fa89bc },
+	{ "recover", 197, 0x4ae5dc12 },
+	{ "stop", 26, 0x936dee2f },
+	{ "resume", 82, 0xb260b7dd },
+	{ "ctx", 29, 0xeb46322d },
+	{ "render", 429, 0xe7477b44 },
+	{ "heaps", 22, 0x85d1b2f7 },
+	{ "teardown", 0, 0x00000000 },
+};
+
+#endif /* PVRSGX_EQUIV_GOLDEN_H */
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_equiv_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_equiv_test.c
new file mode 100644
index 000000000..aa10ee3d3
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_equiv_test.c
@@ -0,0 +1,555 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * The SGX545 path, recorded: every register write, in order, and the bytes
+ * of every structure the kernel builds for the microkernel, through a start,
+ * a command, a recovery, a stop, a resume, a render and a transfer context,
+ * a render target dataset and two frames of a two-kick render built through
+ * it, and the heap query. Included from pvrsgx_fw.c so it reaches the loader's
+ * static functions.
+ *
+ * Registers are a memory buffer, and a stand-in for the microkernel answers
+ * the init kick, takes every command and confirms the power-off. Physical
+ * addresses, which differ between runs, are recorded as the directory they
+ * belong to (PD0, PD1, ...) or as page flags only.
+ *
+ * Each phase's record is checked against pvrsgx_equiv_golden.h: its line
+ * count and CRC32. A change meant to leave the SGX545 alone must leave every
+ * record as it is. On a mismatch the whole record is printed, one "EQ|" line
+ * per entry, to diff against the same test on a known-good tree.
+ */
+
+#include <kunit/test.h>
+
+#include <drm/drm_kunit_helpers.h>
+
+#include <linux/mman.h>
+#include <linux/uaccess.h>
+
+#include "../pvrsgx_job.h"
+#include "../pvrsgx_render.h"
+#include "../pvrsgx_transfer.h"
+#include "../pvrsgx_vm.h"
+
+#include "pvrsgx_equiv_golden.h"
+
+#define EQ_REGS_BYTES	0x10000
+#define EQ_LINE		64
+#define EQ_MAX_LINES	4096
+#define EQ_MAX_PDS	8
+
+struct eq_fx {
+	struct kunit *test;
+	struct pvrsgx_device *sdev;
+	u32 *regs;
+	/* Physical directory addresses, recorded as their index. */
+	u32 pd[EQ_MAX_PDS];
+	unsigned int npd;
+	/* The current phase's record. */
+	char (*line)[EQ_LINE];
+	unsigned int n;
+	bool overflow;
+	/* BIF faults left for the next recovery to find. */
+	unsigned int faults;
+};
+
+/* The fixture of the test that is running, for the write hook. */
+static struct eq_fx *eq_cur;
+
+static __printf(2, 3) void eq_add(struct eq_fx *fx, const char *fmt, ...)
+{
+	va_list ap;
+
+	if (fx->n == EQ_MAX_LINES) {
+		fx->overflow = true;
+		return;
+	}
+	va_start(ap, fmt);
+	vsnprintf(fx->line[fx->n++], EQ_LINE, fmt, ap);
+	va_end(ap);
+}
+
+static void eq_pd(struct eq_fx *fx, struct pvrsgx_mmu *mmu)
+{
+	if (fx->npd < EQ_MAX_PDS)
+		fx->pd[fx->npd++] = pvrsgx_mmu_pd_addr(mmu);
+}
+
+/* A value as recorded: a directory's index, or the value. */
+static const char *eq_val(struct eq_fx *fx, u32 v, char buf[16])
+{
+	for (unsigned int i = 0; i < fx->npd; i++) {
+		if (v == fx->pd[i]) {
+			snprintf(buf, 16, "PD%u", i);
+			return buf;
+		}
+	}
+	snprintf(buf, 16, "%08x", v);
+	return buf;
+}
+
+/* The microkernel's part, answered at once. */
+static void eq_wr(struct pvrsgx_device *sdev, u32 reg, u32 val)
+{
+	struct eq_fx *fx = eq_cur;
+	u32 *hc = sdev->fw.cpu[PVRSGX_FW_HOSTCTL];
+	char b[16];
+
+	eq_add(fx, "wr %04x %s", reg, eq_val(fx, val, b));
+
+	if (reg == sdev->cr->event_kick2 && val == sdev->cr->event_kick2_now)
+		hc[HOSTCTL_INIT_STATUS] |= HOSTCTL_INIT_COMPLETE;
+	if (reg == SGX_CR_EVENT_KICK && val == SGX_CR_EVENT_KICK_NOW) {
+		u32 *ctl = sdev->fw.cpu[PVRSGX_FW_CCBCTL];
+		u32 last = (ctl[0] - 1) & (PVRSGX_CCB_SLOTS - 1);
+		u32 *cmd = (u32 *)sdev->fw.cpu[PVRSGX_FW_CCB] + last * PVRSGX_CCB_CMD_WORDS;
+
+		if (cmd[0] == PVRSGX_KCMD_POWER)
+			hc[HOSTCTL_POWER_STATUS] |= HOSTCTL_POWMAN_POWEROFF_COMPLETE;
+		ctl[1] = ctl[0];
+	}
+	/* The recovery loop clears the events once per fault it handles. */
+	if (reg == SGX_CR_EVENT_HOST_CLEAR && fx->faults && !--fx->faults)
+		fx->regs[SGX_CR_BIF_INT_STAT / 4] = 0;
+}
+
+/* Nonzero words of n bytes at p; word pid_word (if not -1) as "PID". */
+static void eq_words(struct eq_fx *fx, const char *name, const void *p, u32 n, int pid_word)
+{
+	const u32 *w = p;
+	char b[16];
+
+	for (u32 i = 0; i < n / 4; i++) {
+		u32 v = READ_ONCE(w[i]);
+
+		if (!v)
+			continue;
+		if ((int)i == pid_word)
+			eq_add(fx, "%s +%04x PID", name, i * 4);
+		else
+			eq_add(fx, "%s +%04x %s", name, i * 4, eq_val(fx, v, b));
+	}
+}
+
+/* Every valid PDE's flags, and the PTE flags in runs of equal flags. */
+static void eq_mmu(struct eq_fx *fx, const char *name, struct pvrsgx_mmu *mmu)
+{
+	for (u32 d = 0; d < SGX_MMU_PD_ENTRIES; d++) {
+		u32 base = d * SGX_MMU_PDE_SPAN, pde, pte, run = 0, flags = 0, start = 0;
+
+		KUNIT_ASSERT_TRUE(fx->test, pvrsgx_mmu_lookup(mmu, base, &pde, &pte));
+		if (!(pde & SGX_MMU_PDE_VALID))
+			continue;
+		eq_add(fx, "%s pde %03x %x", name, d, (u32)(pde & ~SGX_MMU_ADDR_MASK));
+		for (u32 i = 0; i <= SGX_MMU_PT_ENTRIES; i++) {
+			u32 a = base + i * SGX_MMU_PAGE_SIZE, f = 0;
+
+			if (i < SGX_MMU_PT_ENTRIES) {
+				KUNIT_ASSERT_TRUE(fx->test, pvrsgx_mmu_lookup(mmu, a, &pde, &pte));
+				f = pte & ~SGX_MMU_ADDR_MASK;
+			}
+			if (run && (i == SGX_MMU_PT_ENTRIES || f != flags)) {
+				if (flags)
+					eq_add(fx, "%s pte %08x+%x %x", name, start, run, flags);
+				run = 0;
+			}
+			if (i < SGX_MMU_PT_ENTRIES && !run) {
+				start = a;
+				flags = f;
+			}
+			run++;
+		}
+	}
+}
+
+static void eq_arenas(struct eq_fx *fx)
+{
+	struct pvrsgx_fw *f = &fx->sdev->fw;
+
+	eq_add(fx, "data %08x %x", f->data.dev_addr, f->data.size);
+	eq_words(fx, "data", f->data.cpu, f->data.size, -1);
+	eq_add(fx, "code %08x %x", f->code.dev_addr, f->code.size);
+	eq_words(fx, "code", f->code.cpu, f->code.size, -1);
+	for (int b = 0; b < PVRSGX_FW_NBUF; b++)
+		eq_add(fx, "buf %d %08x", b, f->addr[b]);
+}
+
+/* Close a phase: compare its record with the golden one, print it if they differ. */
+static void eq_check(struct eq_fx *fx, const char *phase)
+{
+	const struct pvrsgx_equiv_golden *g = NULL;
+	u32 crc = ~0u;
+
+	KUNIT_EXPECT_FALSE_MSG(fx->test, fx->overflow, "%s: record overflow", phase);
+	for (unsigned int i = 0; i < fx->n; i++)
+		crc = crc32_le(crc, fx->line[i], strlen(fx->line[i]) + 1);
+	crc = ~crc;
+	for (unsigned int i = 0; i < ARRAY_SIZE(pvrsgx_equiv_golden_545); i++)
+		if (pvrsgx_equiv_golden_545[i].phase &&
+		    !strcmp(pvrsgx_equiv_golden_545[i].phase, phase))
+			g = &pvrsgx_equiv_golden_545[i];
+
+	if (!g || g->lines != fx->n || g->crc != crc) {
+		for (unsigned int i = 0; i < fx->n; i++)
+			kunit_info(fx->test, "EQ| %s %s\n", phase, fx->line[i]);
+		kunit_info(fx->test, "EQ= { \"%s\", %u, 0x%08x },\n", phase, fx->n, crc);
+		KUNIT_FAIL(fx->test, "%s: %u lines, crc 0x%08x; golden %u, 0x%08x", phase, fx->n,
+			   crc, g ? g->lines : 0, g ? g->crc : 0);
+	}
+	fx->n = 0;
+	fx->overflow = false;
+}
+
+/* A microkernel file: three entries and a patch of each kind. */
+#define EQ_IMAGE	0x100
+
+struct eq_file {
+	struct ukfw_header h;
+	struct ukfw_pair entries[3];
+	struct ukfw_pair patches[5];
+	u8 image[EQ_IMAGE];
+};
+
+static struct eq_file *eq_file(struct kunit *test)
+{
+	static const u32 patch[][2] = {
+		{ 0x10, UKFW_TA3DCTL }, { 0x20, UKFW_HOSTCTL }, { 0x30, UKFW_CCBCTL },
+		{ 0x48, UKFW_CCB }, { 0x58, UKFW_MISC },
+	};
+	struct eq_file *t = kunit_kzalloc(test, sizeof(*t), GFP_KERNEL);
+	u32 *w;
+
+	KUNIT_ASSERT_NOT_NULL(test, t);
+	w = (u32 *)t->image;
+	for (u32 i = 0; i < EQ_IMAGE / 4; i++)
+		w[i] = 0x01010101u * (i + 1);
+	for (u32 i = 0; i < ARRAY_SIZE(patch); i++) {
+		w[patch[i][0] / 4] = 0xf80dbeef;
+		w[patch[i][0] / 4 + 1] = 0xfc000000;
+		t->patches[i] = (struct ukfw_pair){ cpu_to_le32(patch[i][0]),
+						    cpu_to_le32(patch[i][1]) };
+	}
+	t->h.magic[0] = cpu_to_le32(UKFW_MAGIC0);
+	t->h.magic[1] = cpu_to_le32(UKFW_MAGIC1);
+	t->h.header_size = cpu_to_le32(offsetof(struct eq_file, image));
+	t->h.interface = cpu_to_le32(1 << 16);
+	t->h.features = cpu_to_le32(PVRSGX_UKFW_CLEANUP_V2 | PVRSGX_UKFW_SECURE |
+				    PVRSGX_UKFW_DONE_SEQ);
+	t->h.image_size = cpu_to_le32(EQ_IMAGE);
+	t->h.image_crc32 = cpu_to_le32(~crc32_le(~0, t->image, EQ_IMAGE));
+	t->h.n_entries = cpu_to_le32(ARRAY_SIZE(t->entries));
+	t->h.entries_off = cpu_to_le32(offsetof(struct eq_file, entries));
+	t->h.n_patches = cpu_to_le32(ARRAY_SIZE(t->patches));
+	t->h.patches_off = cpu_to_le32(offsetof(struct eq_file, patches));
+	t->entries[0] = (struct ukfw_pair){ cpu_to_le32(0), cpu_to_le32(0) };
+	t->entries[1] = (struct ukfw_pair){ cpu_to_le32(2), cpu_to_le32(0x40) };
+	t->entries[2] = (struct ukfw_pair){ cpu_to_le32(4), cpu_to_le32(0x80) };
+	return t;
+}
+
+static void eq_noop_work(struct work_struct *work)
+{
+}
+
+/* pvrsgx_device_init() up to the start, with the firmware from eq_file(). */
+static struct eq_fx *eq_device(struct kunit *test, unsigned int core, u32 core_rev)
+{
+	struct eq_fx *fx = kunit_kzalloc(test, sizeof(*fx), GFP_KERNEL);
+	struct pvrsgx_device *sdev;
+	struct firmware fw = {};
+	struct eq_file *file;
+	struct device *dev;
+
+	KUNIT_ASSERT_NOT_NULL(test, fx);
+	fx->test = test;
+	fx->line = kunit_kcalloc(test, EQ_MAX_LINES, EQ_LINE, GFP_KERNEL);
+	fx->regs = kunit_kzalloc(test, EQ_REGS_BYTES, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_NULL(test, fx->line);
+	KUNIT_ASSERT_NOT_NULL(test, fx->regs);
+
+	dev = drm_kunit_helper_alloc_device(test);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, dev);
+	sdev = drm_kunit_helper_alloc_drm_device(test, dev, struct pvrsgx_device, base, 0);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sdev);
+	fx->sdev = sdev;
+
+	mutex_init(&sdev->ccb_lock);
+	mutex_init(&sdev->done_lock);
+	spin_lock_init(&sdev->ccb_spin);
+	spin_lock_init(&sdev->inval_lock);
+	INIT_LIST_HEAD(&sdev->inval_waits);
+	INIT_WORK(&sdev->reap_work, eq_noop_work);
+	/* The restart path queues the reap; the real driver's own workqueue isn't set up here. */
+	sdev->wq = system_dfl_wq;
+	sdev->free_wq = system_dfl_wq;
+	sdev->irq = -1;
+	sdev->regs = (void __iomem __force *)fx->regs;
+
+	sdev->family = pvrsgx_core_family_get(core);
+	KUNIT_ASSERT_NOT_NULL(test, sdev->family);
+	sdev->cr = sdev->family->regs;
+	sdev->core_id = sdev->family->core_id;
+	sdev->core_rev = core_rev;
+	sdev->core = pvrsgx_core_lookup(sdev->family, sdev->core_id, sdev->core_rev);
+	KUNIT_ASSERT_NOT_NULL(test, sdev->core);
+
+	sdev->kernel_mmu = pvrsgx_mmu_create(sdev);
+	sdev->idle_mmu = pvrsgx_mmu_create(sdev);
+	sdev->reset_mmu = pvrsgx_mmu_create(sdev);
+	sdev->reset_page = alloc_page(GFP_KERNEL | __GFP_ZERO | __GFP_DMA32);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sdev->kernel_mmu);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sdev->idle_mmu);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sdev->reset_mmu);
+	KUNIT_ASSERT_NOT_NULL(test, sdev->reset_page);
+	eq_pd(fx, sdev->kernel_mmu);
+	eq_pd(fx, sdev->idle_mmu);
+	eq_pd(fx, sdev->reset_mmu);
+
+	pvrsgx_kobj_init(sdev);
+	mutex_init(&sdev->pb_lock);
+
+	file = eq_file(test);
+	fw.data = (const u8 *)file;
+	fw.size = sizeof(*file);
+	ida_init(&sdev->fw.pd_slots);
+	sdev->fw.name = "equiv";
+	KUNIT_ASSERT_EQ(test, ukfw_load(sdev, &fw), 0);
+	KUNIT_ASSERT_EQ(test, fw_arenas(sdev), 0);
+
+	eq_cur = fx;
+	sdev->test_wr = eq_wr;
+	return fx;
+}
+
+static void eq_fini(struct eq_fx *fx)
+{
+	struct pvrsgx_device *sdev = fx->sdev;
+
+	sdev->test_wr = NULL;
+	eq_cur = NULL;
+	cancel_work_sync(&sdev->reap_work);
+	pvrsgx_ccb_inval_reap(sdev, true);
+	pvrsgx_pb_put(sdev->pb);
+	sdev->pb = NULL;
+	pvrsgx_kobj_fini(sdev);
+	pvrsgx_fw_fini(sdev);
+	__free_page(sdev->reset_page);
+	pvrsgx_mmu_destroy(sdev->reset_mmu);
+	pvrsgx_mmu_destroy(sdev->idle_mmu);
+	pvrsgx_mmu_destroy(sdev->kernel_mmu);
+	mutex_destroy(&sdev->pb_lock);
+	mutex_destroy(&sdev->done_lock);
+	mutex_destroy(&sdev->ccb_lock);
+}
+
+/* DEV_QUERY_HEAP_INFO_GET, through user memory as a client sees it. */
+static void eq_heaps(struct eq_fx *fx)
+{
+	struct drm_powervr_sgx_dev_query_heap_info info = {};
+	struct drm_powervr_sgx_heap heaps[16] = {};
+	unsigned long user;
+	u32 size = sizeof(info);
+
+	user = kunit_vm_mmap(fx->test, NULL, 0, PAGE_SIZE, PROT_READ | PROT_WRITE,
+			     MAP_ANONYMOUS | MAP_PRIVATE, 0);
+	KUNIT_ASSERT_NE_MSG(fx->test, user, 0, "no user memory");
+	info.heaps.stride = sizeof(heaps[0]);
+	info.heaps.count = ARRAY_SIZE(heaps);
+	info.heaps.array = user + 64;
+	KUNIT_ASSERT_EQ(fx->test, copy_to_user((void __user *)user, &info, sizeof(info)), 0);
+	KUNIT_ASSERT_EQ(fx->test, pvrsgx_query_heap_info(fx->sdev, (void __user *)user, &size), 0);
+	KUNIT_ASSERT_EQ(fx->test, copy_from_user(&info, (void __user *)user, sizeof(info)), 0);
+	KUNIT_ASSERT_EQ(fx->test, copy_from_user(heaps, (void __user *)(user + 64),
+						 sizeof(heaps)), 0);
+	eq_add(fx, "heaps size %u stride %u count %u", size, info.heaps.stride, info.heaps.count);
+	for (u32 i = 0; i < min_t(u32, info.heaps.count, ARRAY_SIZE(heaps)); i++) {
+		char name[16];
+
+		snprintf(name, sizeof(name), "heap%u", i);
+		eq_words(fx, name, &heaps[i], sizeof(heaps[i]), -1);
+	}
+}
+
+/*
+ * A 720x480 dataset, then two frames of a two-kick render: everything the
+ * kernel writes for the microkernel at the dataset's creation and at each
+ * kick (the PB, the dataset's structures, the TA commands, the render
+ * details, the background object and the syncs).
+ */
+static void eq_render(struct eq_fx *fx, struct pvrsgx_context *ctx)
+{
+	struct pvrsgx_device *sdev = fx->sdev;
+	struct pvrsgx_render_ctx *r = ctx->render;
+	u64 pds = pvrsgx_heaps(sdev)->pds_pixel.base;
+	struct pvrsgx_rt_dataset *s;
+	struct pvrsgx_job *job;
+	char name[24];
+
+	eq_words(fx, "pbdesc", kobj_cpu(r->pb->desc), sizeof(struct pvrsgx_fw_pb_desc), -1);
+	eq_words(fx, "pbblocks", kobj_cpu(r->pb->blocks), 12, -1);
+	eq_words(fx, "pbfl", kobj_cpu(r->pb->freelist), r->pb->freelist->km.size, -1);
+
+	s = pvrsgx_rtds_create(ctx, 720, 480, 1);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(fx->test, s);
+	eq_add(fx, "rtds rtword %08x rtdata %08x done %08x tailptr %08x", kobj_va(s->rtword),
+	       kobj_va(s->rtdata), kobj_va(s->done), kobj_va(s->tailptr));
+	eq_words(fx, "rtdata", kobj_cpu(s->rtdata), sizeof(struct pvrsgx_fw_rt_data), -1);
+	for (int i = 0; i < 2; i++) {
+		snprintf(name, sizeof(name), "rd%d", i);
+		eq_words(fx, name, kobj_cpu(s->rd[i]), sizeof(struct pvrsgx_fw_render_details), -1);
+		snprintf(name, sizeof(name), "bg%d", i);
+		eq_words(fx, name, kobj_cpu(s->bgblock[i]), 0x60, -1);
+		snprintf(name, sizeof(name), "hwsync%d", i);
+		eq_words(fx, name, kobj_cpu(s->hwsync[i]), sizeof(struct pvrsgx_fw_rt_deps), -1);
+	}
+
+	job = kunit_kzalloc(fx->test, sizeof(*job), GFP_KERNEL);
+	KUNIT_ASSERT_NOT_NULL(fx->test, job);
+	job->ctx = ctx;
+	job->render.rtds = s;
+	job->render.nkicks = 2;
+	job->render.render.base.depth_addr = 0x20100000;
+	job->render.render.base.stencil_addr = 0x20180000;
+	job->render.render.base.pixel_event_pds_addr = pds + 0x300;
+	job->render.render.base.pixel_event_pds_rows = 3;
+	job->render.render.base.bg_pds_secondary_addr = pds + 0x200;
+	job->render.render.base.bg_pds_secondary_rows = 2;
+	job->render.render.base.bg_pds_primary_addr = pds + 0x100;
+	job->render.render.base.bg_pds_primary_rows = 4;
+	job->render.render.base.bg_depth_f32 = 0x3f800000;
+	job->render.render.base.depth_bias_factor_f32 = 0x3f000000;
+	job->render.render.sgx545_render.zls_control = 0x00171004;
+	job->render.kick[0].base.vdm_base = 0x20000000;
+	job->render.kick[1].base.vdm_base = 0x20001000;
+	job->render.kick[1].sgx545_ta_kick.ta_flags2 = 0x1;
+
+	for (u32 f = 0; f < 2; f++) {
+		for (u32 k = 0; k < 2; k++) {
+			u32 woff = r->ccb_woff, data[6];
+
+			KUNIT_ASSERT_EQ(fx->test, pvrsgx_render_job_build(job, k, data), 0);
+			snprintf(name, sizeof(name), "f%u k%u ta", f, k);
+			eq_words(fx, name, (u8 *)kobj_cpu(r->ccb) + woff,
+				 sizeof(struct pvrsgx_fw_ta_cmd), -1);
+			eq_add(fx, "f%u k%u data %08x %08x woff %x", f, k, data[0], data[1],
+			       r->ccb_woff);
+		}
+		eq_add(fx, "f%u entry %u cmd_off %x done_val %x", f, job->render.entry,
+		       job->render.cmd_off, job->done_val);
+		snprintf(name, sizeof(name), "f%u rd", f);
+		eq_words(fx, name, kobj_cpu(s->rd[job->render.entry]),
+			 sizeof(struct pvrsgx_fw_render_details), -1);
+		snprintf(name, sizeof(name), "f%u bg", f);
+		eq_words(fx, name, kobj_cpu(s->bgblock[job->render.entry]), 0x60, -1);
+		snprintf(name, sizeof(name), "f%u w4", f);
+		eq_words(fx, name, kobj_cpu(s->rd_w4[job->render.entry]), 4, -1);
+		snprintf(name, sizeof(name), "f%u deps", f);
+		eq_words(fx, name, kobj_cpu(s->hwsync[job->render.entry]),
+			 sizeof(struct pvrsgx_fw_rt_deps), -1);
+		eq_words(fx, "rtword", kobj_cpu(s->rtword), 4, -1);
+		eq_words(fx, "sync3d", kobj_cpu(r->sync3d), SYNC_DATA_BYTES, -1);
+		eq_words(fx, "done", kobj_cpu(s->done), SYNC_DATA_BYTES, -1);
+	}
+	pvrsgx_rtds_put(s);
+}
+
+static void equiv_test_sgx545(struct kunit *test)
+{
+	struct eq_fx *fx = eq_device(test, 545, 0x0001000e);
+	struct pvrsgx_device *sdev = fx->sdev;
+	struct pvrsgx_context rctx = {}, xctx = {};
+	u32 data[6] = { 0x11, 0x22, 0x33, 0x44, 0x55, 0x66 };
+	struct pvrsgx_vm *vm;
+
+	/* The image built, the arenas mapped. */
+	eq_arenas(fx);
+	eq_mmu(fx, "kmmu", sdev->kernel_mmu);
+	eq_check(fx, "init");
+
+	/* The first start. */
+	KUNIT_ASSERT_EQ(test, pvrsgx_fw_start(sdev), 0);
+	eq_words(fx, "data", sdev->fw.data.cpu, sdev->fw.data.size, -1);
+	eq_check(fx, "start");
+
+	/* A command that asks for every invalidate. */
+	sdev->fw.cache_control |= PVRSGX_CC_INVAL_BIF_PT | PVRSGX_CC_INVAL_BIF_PD |
+				  PVRSGX_CC_INVAL_DATA;
+	KUNIT_ASSERT_EQ(test, pvrsgx_ccb_send(sdev, PVRSGX_KCMD_PROCESS_QUEUES, data, NULL), 0);
+	eq_words(fx, "data", sdev->fw.data.cpu, 0x3200, -1);
+	eq_check(fx, "send");
+
+	/* A hang: the restart finds a BIF fault left over. */
+	fx->regs[SGX_CR_BIF_INT_STAT / 4] = sdev->cr->bif_int_stat_fault_req;
+	fx->regs[SGX_CR_BIF_FAULT / 4] = 0x3e001000;
+	fx->faults = 1;
+	sdev->fw_running = false;
+	KUNIT_ASSERT_EQ(test, pvrsgx_fw_start(sdev), 0);
+	eq_arenas(fx);
+	eq_mmu(fx, "rmmu", sdev->reset_mmu);
+	eq_check(fx, "recover");
+
+	/* Runtime suspend. */
+	pvrsgx_fw_stop(sdev);
+	eq_words(fx, "data", sdev->fw.data.cpu, 0x3200, -1);
+	eq_check(fx, "stop");
+
+	/* Runtime resume, from gated clocks. */
+	KUNIT_ASSERT_EQ(test, pvrsgx_fw_start(sdev), 0);
+	eq_words(fx, "data", sdev->fw.data.cpu, sdev->fw.data.size, -1);
+	eq_check(fx, "resume");
+
+	/* A VM with a render and a transfer context. */
+	vm = pvrsgx_vm_create(sdev);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, vm);
+	eq_pd(fx, vm->mmu);
+	kref_init(&rctx.ref);
+	rctx.sdev = sdev;
+	rctx.vm = vm;
+	xctx.sdev = sdev;
+	xctx.vm = vm;
+	KUNIT_ASSERT_EQ(test, pvrsgx_render_ctx_init(sdev, &rctx, 0x40), 0);
+	KUNIT_ASSERT_EQ(test, pvrsgx_transfer_ctx_init(sdev, &xctx), 0);
+	eq_add(fx, "rctx hw %08x ccb %08x ccbctl %08x", kobj_va(rctx.render->hwctx),
+	       kobj_va(rctx.render->ccb), kobj_va(rctx.render->ccbctl));
+	eq_add(fx, "rctx scratch %08x sync %08x", kobj_va(rctx.render->scratch),
+	       kobj_va(rctx.render->sync3d));
+	eq_words(fx, "rhw", kobj_cpu(rctx.render->hwctx), sizeof(struct pvrsgx_fw_hw_render_ctx),
+		 offsetof(struct pvrsgx_fw_hw_render_ctx, pid) / 4);
+	eq_add(fx, "pb pages %x base %08x buf %08x fl %08x", rctx.render->pb->pages,
+	       rctx.render->pb->base, kobj_va(rctx.render->pb->buf),
+	       kobj_va(rctx.render->pb->freelist));
+	eq_add(fx, "pb desc %08x blocks %08x", kobj_va(rctx.render->pb->desc),
+	       kobj_va(rctx.render->pb->blocks));
+	eq_add(fx, "xctx hw %08x", kobj_va(xctx.transfer->hwctx));
+	eq_words(fx, "xhw", kobj_cpu(xctx.transfer->hwctx),
+		 sizeof(struct pvrsgx_fw_hw_transfer_ctx),
+		 offsetof(struct pvrsgx_fw_hw_transfer_ctx, pid) / 4);
+	eq_mmu(fx, "vmmu", vm->mmu);
+	eq_mmu(fx, "kmmu", sdev->kernel_mmu);
+	eq_check(fx, "ctx");
+
+	eq_render(fx, &rctx);
+	eq_check(fx, "render");
+
+	eq_heaps(fx);
+	eq_check(fx, "heaps");
+
+	pvrsgx_transfer_ctx_fini(sdev, &xctx);
+	pvrsgx_render_ctx_fini(sdev, &rctx);
+	pvrsgx_vm_put(vm);
+	eq_check(fx, "teardown");
+	eq_fini(fx);
+}
+
+static struct kunit_case pvrsgx_equiv_test_cases[] = {
+	KUNIT_CASE(equiv_test_sgx545),
+	{}
+};
+
+static struct kunit_suite pvrsgx_equiv_test_suite = {
+	.name = "powervr_sgx_equiv",
+	.test_cases = pvrsgx_equiv_test_cases,
+};
+
+kunit_test_suite(pvrsgx_equiv_test_suite);
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_fw_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_fw_test.c
new file mode 100644
index 000000000..a278cdc41
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_fw_test.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * KUnit tests of the microkernel file's checks,
+ * included from pvrsgx_fw.c so they reach its static functions.
+ */
+
+#include <kunit/test.h>
+
+#define T_IMAGE		256
+#define T_ENTRIES	2
+#define T_PATCHES	2
+#define T_SIZE		(UKFW_HEADER_MIN + 8 * (T_ENTRIES + T_PATCHES) + T_IMAGE)
+
+struct ukfw_test_file {
+	struct ukfw_header h;
+	struct ukfw_pair entries[T_ENTRIES];
+	struct ukfw_pair patches[T_PATCHES];
+	u8 image[T_IMAGE];
+};
+
+static_assert(sizeof(struct ukfw_header) == UKFW_HEADER_MIN);
+static_assert(sizeof(struct ukfw_test_file) == T_SIZE);
+
+/* A good file: event entry at 0, two patches, the image a counting pattern. */
+static struct ukfw_test_file *ukfw_good(struct kunit *test)
+{
+	struct ukfw_test_file *t = kunit_kzalloc(test, sizeof(*t), GFP_KERNEL);
+
+	KUNIT_ASSERT_NOT_NULL(test, t);
+	for (u32 i = 0; i < T_IMAGE; i++)
+		t->image[i] = i;
+	t->h.magic[0] = cpu_to_le32(UKFW_MAGIC0);
+	t->h.magic[1] = cpu_to_le32(UKFW_MAGIC1);
+	t->h.header_size = cpu_to_le32(offsetof(struct ukfw_test_file, image));
+	t->h.interface = cpu_to_le32(1 << 16);
+	t->h.features = cpu_to_le32(PVRSGX_UKFW_CLEANUP_V2);
+	t->h.image_size = cpu_to_le32(T_IMAGE);
+	t->h.n_entries = cpu_to_le32(T_ENTRIES);
+	t->h.entries_off = cpu_to_le32(offsetof(struct ukfw_test_file, entries));
+	t->h.n_patches = cpu_to_le32(T_PATCHES);
+	t->h.patches_off = cpu_to_le32(offsetof(struct ukfw_test_file, patches));
+	t->entries[0] = (struct ukfw_pair){ cpu_to_le32(0), cpu_to_le32(0) };
+	t->entries[1] = (struct ukfw_pair){ cpu_to_le32(2), cpu_to_le32(0x40) };
+	t->patches[0] = (struct ukfw_pair){ cpu_to_le32(8), cpu_to_le32(UKFW_TA3DCTL) };
+	t->patches[1] = (struct ukfw_pair){ cpu_to_le32(0x48), cpu_to_le32(UKFW_HOSTCTL) };
+	return t;
+}
+
+static void ukfw_seal(struct ukfw_test_file *t)
+{
+	t->h.image_crc32 = cpu_to_le32(~crc32_le(~0, t->image, T_IMAGE));
+}
+
+static int ukfw_try(struct kunit *test, struct ukfw_test_file *t, size_t size)
+{
+	struct pvrsgx_fw f = {};
+	const char *why = "";
+	int err = ukfw_parse((const u8 *)t, size, &f, &why);
+
+	kunit_info(test, "%d %s\n", err, err == -EINVAL ? why : "");
+	if (!err) {
+		KUNIT_EXPECT_EQ(test, f.image_size, T_IMAGE);
+		KUNIT_EXPECT_EQ(test, f.features, PVRSGX_UKFW_CLEANUP_V2);
+		KUNIT_EXPECT_EQ(test, f.n_entries, T_ENTRIES);
+		KUNIT_EXPECT_EQ(test, f.entries[1][0], 2);
+		KUNIT_EXPECT_EQ(test, f.entries[1][1], 0x40);
+		KUNIT_EXPECT_EQ(test, f.patches[1][1], UKFW_HOSTCTL);
+		KUNIT_EXPECT_MEMEQ(test, f.image, t->image, T_IMAGE);
+	}
+	kfree(f.entries);
+	kfree(f.patches);
+	kfree(f.image);
+	return err;
+}
+
+static void ukfw_test_good(struct kunit *test)
+{
+	struct ukfw_test_file *t = ukfw_good(test);
+
+	ukfw_seal(t);
+	KUNIT_EXPECT_EQ(test, ukfw_try(test, t, T_SIZE), 0);
+}
+
+static void ukfw_test_not_ours(struct kunit *test)
+{
+	struct ukfw_test_file *t = ukfw_good(test);
+
+	ukfw_seal(t);
+	t->h.magic[1] = 0;
+	KUNIT_EXPECT_EQ(test, ukfw_try(test, t, T_SIZE), -ENOENT);
+	KUNIT_EXPECT_EQ(test, ukfw_try(test, t, UKFW_HEADER_MIN - 1), -ENOENT);
+}
+
+static void ukfw_test_crc(struct kunit *test)
+{
+	struct ukfw_test_file *t = ukfw_good(test);
+
+	ukfw_seal(t);
+	t->image[100] ^= 1;
+	KUNIT_EXPECT_EQ(test, ukfw_try(test, t, T_SIZE), -EINVAL);
+}
+
+static void ukfw_test_header(struct kunit *test)
+{
+	static const struct { u32 off, val; } bad[] = {
+		{ offsetof(struct ukfw_header, interface), 2 << 16 },	/* unknown major */
+		{ offsetof(struct ukfw_header, features), BIT(7) },	/* unknown feature */
+		{ offsetof(struct ukfw_header, header_size), 0x58 },	/* below the minimum */
+		{ offsetof(struct ukfw_header, header_size), 0x84 },	/* not 8-aligned */
+		{ offsetof(struct ukfw_header, image_size), 0 },
+		{ offsetof(struct ukfw_header, image_size), T_IMAGE + 8 },	/* past the end */
+		{ offsetof(struct ukfw_header, image_size), 0x1001 },	/* not 8-aligned */
+		{ offsetof(struct ukfw_header, n_entries), 0 },		/* no event entry */
+		{ offsetof(struct ukfw_header, n_entries), 65 },
+		{ offsetof(struct ukfw_header, entries_off), T_SIZE - 8 },	/* past the end */
+		{ offsetof(struct ukfw_header, patches_off), 2 },	/* misaligned table */
+	};
+
+	for (u32 i = 0; i < ARRAY_SIZE(bad); i++) {
+		struct ukfw_test_file *t = ukfw_good(test);
+
+		*(__le32 *)((u8 *)&t->h + bad[i].off) = cpu_to_le32(bad[i].val);
+		ukfw_seal(t);
+		KUNIT_EXPECT_EQ_MSG(test, ukfw_try(test, t, T_SIZE), -EINVAL, "case %u", i);
+	}
+}
+
+static void ukfw_test_tables(struct kunit *test)
+{
+	static const struct { bool entry; u32 idx, a, b; } bad[] = {
+		{ true, 0, 1, 0 },		/* no program 0 left */
+		{ true, 1, 5, 0x40 },		/* program out of range */
+		{ true, 1, 2, T_IMAGE },	/* entry past the image */
+		{ true, 1, 2, 0x44 },		/* entry not on an instruction */
+		{ false, 0, T_IMAGE - 4, UKFW_TA3DCTL },	/* patch past the image */
+		{ false, 0, 0xc, UKFW_TA3DCTL },	/* patch not on an instruction */
+		{ false, 1, 0x48, 0 },		/* value 0 */
+		{ false, 1, 0x48, UKFW_PDTABLE + 1 },	/* unknown value */
+	};
+
+	for (u32 i = 0; i < ARRAY_SIZE(bad); i++) {
+		struct ukfw_test_file *t = ukfw_good(test);
+		struct ukfw_pair *p = bad[i].entry ? &t->entries[bad[i].idx] :
+						     &t->patches[bad[i].idx];
+
+		*p = (struct ukfw_pair){ cpu_to_le32(bad[i].a), cpu_to_le32(bad[i].b) };
+		ukfw_seal(t);
+		KUNIT_EXPECT_EQ_MSG(test, ukfw_try(test, t, T_SIZE), -EINVAL, "case %u", i);
+	}
+}
+
+static struct kunit_case pvrsgx_fw_test_cases[] = {
+	KUNIT_CASE(ukfw_test_good),
+	KUNIT_CASE(ukfw_test_not_ours),
+	KUNIT_CASE(ukfw_test_crc),
+	KUNIT_CASE(ukfw_test_header),
+	KUNIT_CASE(ukfw_test_tables),
+	{}
+};
+
+static struct kunit_suite pvrsgx_fw_test_suite = {
+	.name = "powervr_sgx_fw",
+	.test_cases = pvrsgx_fw_test_cases,
+};
+
+kunit_test_suite(pvrsgx_fw_test_suite);
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_kobj_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_kobj_test.c
new file mode 100644
index 000000000..59de4da9f
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_kobj_test.c
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * KUnit tests of kernel-object teardown at unbind, included from
+ * pvrsgx_render.c so they reach the device PB's static functions. No GPU:
+ * the objects, page tables and VMs are memory only.
+ */
+
+#include <kunit/test.h>
+
+#include <drm/drm_kunit_helpers.h>
+
+/* The parts of pvrsgx_device_init() that kernel objects, VMs and PBs need. */
+static struct pvrsgx_device *kobj_test_device(struct kunit *test)
+{
+	struct pvrsgx_device *sdev;
+	struct device *dev;
+
+	dev = drm_kunit_helper_alloc_device(test);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, dev);
+	sdev = drm_kunit_helper_alloc_drm_device(test, dev, struct pvrsgx_device, base, 0);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sdev);
+	/* The kernel object ranges come from the core's layout. */
+	sdev->family = pvrsgx_core_family_get(545);
+	KUNIT_ASSERT_NOT_NULL(test, sdev->family);
+	sdev->cr = sdev->family->regs;
+	sdev->kernel_mmu = pvrsgx_mmu_create(sdev);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, sdev->kernel_mmu);
+	mutex_init(&sdev->ccb_lock);
+	spin_lock_init(&sdev->ccb_spin);
+	spin_lock_init(&sdev->inval_lock);
+	INIT_LIST_HEAD(&sdev->inval_waits);
+	mutex_init(&sdev->pb_lock);
+	pvrsgx_kobj_init(sdev);
+	return sdev;
+}
+
+/* What the device release does with them; every range must come back empty. */
+static void kobj_test_fini(struct kunit *test, struct pvrsgx_device *sdev)
+{
+	pvrsgx_pb_put(sdev->pb);
+	sdev->pb = NULL;
+	pvrsgx_kobj_fini(sdev);
+	KUNIT_EXPECT_TRUE(test, list_empty(&sdev->graveyard));
+	KUNIT_EXPECT_TRUE(test, list_empty(&sdev->kobj_unmapped));
+	for (int r = 0; r < PVRSGX_KRANGE_COUNT; r++)
+		KUNIT_EXPECT_TRUE_MSG(test, drm_mm_clean(&sdev->krange[r]), "range %d", r);
+	pvrsgx_mmu_destroy(sdev->kernel_mmu);
+	mutex_destroy(&sdev->pb_lock);
+	mutex_destroy(&sdev->ccb_lock);
+}
+
+/*
+ * A client's last render context goes, and no render runs after it: its
+ * per-VM scratch object waits in the graveyard and keeps the VM, so the VM
+ * and its PB reference are freed only at unbind, by the reap. The PB's
+ * objects join the graveyard then, and unbind must free them too.
+ */
+static void kobj_test_vm_held_by_graveyard(struct kunit *test)
+{
+	struct pvrsgx_device *sdev = kobj_test_device(test);
+	struct pvrsgx_kobj *scratch = NULL;
+	struct pvrsgx_vm *vm;
+	struct pvrsgx_pb *pb;
+
+	vm = pvrsgx_vm_create(sdev);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, vm);
+	pb = dev_pb_get(sdev, 0x40);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pb);
+	KUNIT_ASSERT_EQ(test, pvrsgx_pb_map(vm, pb), 0);
+	KUNIT_ASSERT_EQ(test, pvrsgx_kobj_new(sdev, &scratch, PVRSGX_KRANGE_3DPARAMS, 0x40,
+					      pb->base, vm), 0);
+
+	/* pvrsgx_render_ctx_fini(), then the file's VM handle. */
+	pvrsgx_kobj_drop(sdev, &scratch);
+	pvrsgx_pb_put(pb);
+	pvrsgx_vm_put(vm);
+	KUNIT_EXPECT_FALSE(test, list_empty(&sdev->graveyard));
+
+	kobj_test_fini(test, sdev);
+}
+
+/* The VM goes first and nothing waits: the PB is the device's alone at unbind. */
+static void kobj_test_vm_gone_first(struct kunit *test)
+{
+	struct pvrsgx_device *sdev = kobj_test_device(test);
+	struct pvrsgx_vm *vm;
+	struct pvrsgx_pb *pb;
+
+	vm = pvrsgx_vm_create(sdev);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, vm);
+	pb = dev_pb_get(sdev, 0x40);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pb);
+	KUNIT_ASSERT_EQ(test, pvrsgx_pb_map(vm, pb), 0);
+	pvrsgx_pb_put(pb);
+	pvrsgx_vm_put(vm);
+	KUNIT_EXPECT_TRUE(test, list_empty(&sdev->graveyard));
+
+	kobj_test_fini(test, sdev);
+}
+
+/*
+ * A reaped object is unmapped at once, but its pages and address wait until
+ * the microkernel has taken a command after it: that command's cache control
+ * invalidates the BIF TLB, which may still translate the old pages.
+ */
+static void kobj_test_free_after_invalidate(struct kunit *test)
+{
+	struct pvrsgx_device *sdev = kobj_test_device(test);
+	u32 *ccbctl = kunit_kzalloc(test, 2 * sizeof(u32), GFP_KERNEL);
+	struct pvrsgx_kobj *o = NULL;
+	u32 addr;
+
+	KUNIT_ASSERT_NOT_NULL(test, ccbctl);
+	/* An empty ring: everything issued is consumed. */
+	sdev->fw.cpu[PVRSGX_FW_CCBCTL] = ccbctl;
+	KUNIT_ASSERT_EQ(test, pvrsgx_kobj_new(sdev, &o, PVRSGX_KRANGE_KDATA, PAGE_SIZE, 0, NULL),
+			0);
+	addr = kobj_va(o);
+	sdev->fw.cache_control = 0;
+
+	/* Released before render 1, which has finished. */
+	pvrsgx_kobj_drop(sdev, &o);
+	pvrsgx_kobj_reap(sdev, 1);
+	KUNIT_EXPECT_TRUE(test, list_empty(&sdev->graveyard));
+	KUNIT_EXPECT_FALSE(test, list_empty(&sdev->kobj_unmapped));
+	KUNIT_EXPECT_TRUE(test, pvrsgx_mmu_range_empty(sdev->kernel_mmu, addr, PAGE_SIZE));
+	KUNIT_EXPECT_EQ(test, sdev->fw.cache_control & PVRSGX_CC_INVAL_BIF_PT,
+			PVRSGX_CC_INVAL_BIF_PT);
+	KUNIT_EXPECT_FALSE(test, drm_mm_clean(&sdev->krange[PVRSGX_KRANGE_KDATA]));
+
+	/* No command since: still held. */
+	pvrsgx_kobj_reap(sdev, 1);
+	KUNIT_EXPECT_FALSE(test, list_empty(&sdev->kobj_unmapped));
+
+	/* The next command was sent and taken. */
+	sdev->ccb_issued++;
+	pvrsgx_kobj_reap(sdev, 1);
+	KUNIT_EXPECT_TRUE(test, list_empty(&sdev->kobj_unmapped));
+	KUNIT_EXPECT_TRUE(test, drm_mm_clean(&sdev->krange[PVRSGX_KRANGE_KDATA]));
+
+	sdev->fw.cpu[PVRSGX_FW_CCBCTL] = NULL;
+	kobj_test_fini(test, sdev);
+}
+
+struct kobj_test_wait {
+	struct pvrsgx_inval_wait w;
+	bool done;
+};
+
+static void kobj_test_wait_done(struct pvrsgx_inval_wait *w)
+{
+	container_of(w, struct kobj_test_wait, w)->done = true;
+}
+
+/* What an unmap freed waits for the command that carries the invalidate. */
+static void kobj_test_after_invalidate(struct kunit *test)
+{
+	struct pvrsgx_device *sdev = kobj_test_device(test);
+	u32 *ccbctl = kunit_kzalloc(test, 2 * sizeof(u32), GFP_KERNEL);
+	struct kobj_test_wait t = { .w.done = kobj_test_wait_done };
+
+	KUNIT_ASSERT_NOT_NULL(test, ccbctl);
+	sdev->fw.cpu[PVRSGX_FW_CCBCTL] = ccbctl;
+	sdev->fw.cache_control = 0;
+
+	pvrsgx_ccb_after_invalidate(sdev, &t.w, PVRSGX_CC_INVAL_BIF_PT | PVRSGX_CC_INVAL_BIF_PD);
+	KUNIT_EXPECT_EQ(test, sdev->fw.cache_control,
+			PVRSGX_CC_INVAL_BIF_PT | PVRSGX_CC_INVAL_BIF_PD);
+	pvrsgx_ccb_inval_reap(sdev, false);
+	KUNIT_EXPECT_FALSE(test, t.done);
+
+	/* The next command was sent and taken. */
+	sdev->ccb_issued++;
+	pvrsgx_ccb_inval_reap(sdev, false);
+	KUNIT_EXPECT_TRUE(test, t.done);
+	KUNIT_EXPECT_TRUE(test, list_empty(&sdev->inval_waits));
+
+	/* Once the GPU is lost, nothing waits. */
+	t.done = false;
+	WRITE_ONCE(sdev->lost, true);
+	pvrsgx_ccb_after_invalidate(sdev, &t.w, PVRSGX_CC_INVAL_BIF_PT);
+	KUNIT_EXPECT_TRUE(test, t.done);
+
+	sdev->fw.cpu[PVRSGX_FW_CCBCTL] = NULL;
+	kobj_test_fini(test, sdev);
+}
+
+static struct kunit_case pvrsgx_kobj_test_cases[] = {
+	KUNIT_CASE(kobj_test_vm_held_by_graveyard),
+	KUNIT_CASE(kobj_test_vm_gone_first),
+	KUNIT_CASE(kobj_test_free_after_invalidate),
+	KUNIT_CASE(kobj_test_after_invalidate),
+	{}
+};
+
+static struct kunit_suite pvrsgx_kobj_test_suite = {
+	.name = "powervr_sgx_kobj",
+	.test_cases = pvrsgx_kobj_test_cases,
+};
+
+kunit_test_suite(pvrsgx_kobj_test_suite);
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_prime_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_prime_test.c
new file mode 100644
index 000000000..9eb7faab4
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_prime_test.c
@@ -0,0 +1,125 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/* KUnit tests of a job's cache ranges and their checks, included from pvrsgx_prime.c. */
+
+#include <kunit/test.h>
+
+#include <linux/err.h>
+#include <linux/mman.h>
+#include <linux/uaccess.h>
+
+#define R(f, o, sz, st, n) ((struct drm_powervr_sgx_cache_range){ .handle = 1, .flags = (f), \
+	.offset = (o), .size = (sz), .stride = (st), .rows = (n) })
+#define RD DRM_POWERVR_SGX_CACHE_RANGE_FLAG_READ
+#define WR DRM_POWERVR_SGX_CACHE_RANGE_FLAG_WRITE
+
+static void prime_test_range_accepts(struct kunit *test)
+{
+	const u64 obj = 2944 * 480;
+	struct drm_powervr_sgx_cache_range r;
+
+	/* a linear range, the whole object, rows 0 and 1 alike */
+	r = R(RD, 0, obj, 0, 0);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), 0);
+	r = R(WR, 0, obj, 0, 1);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), 0);
+	/* a 64x16 damage rect at (100, 200) of a 720x480 XRGB buffer, pitch 2944 */
+	r = R(RD | WR, 200 * 2944 + 100 * 4, 64 * 4, 2944, 16);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), 0);
+	/* the last row ends at the object's end exactly */
+	r = R(WR, 0, 720 * 4, 2944, 480);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), 0);
+	r = R(WR, 2944 - 720 * 4, 720 * 4, 2944, 480);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), 0);
+	/* rows exactly stride apart are one span */
+	r = R(WR, 0, 2944, 2944, 480);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), 0);
+}
+
+static void prime_test_range_refuses(struct kunit *test)
+{
+	const u64 obj = 2944 * 480;
+	struct drm_powervr_sgx_cache_range r;
+
+	/* flags: none, or an unknown bit */
+	r = R(0, 0, 64, 0, 1);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	r = R(RD | BIT(2), 0, 64, 0, 1);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	/* an empty row */
+	r = R(RD, 0, 0, 0, 1);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	/* rows overlapping: stride below size */
+	r = R(RD, 0, 64, 32, 2);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	/* one byte past the end, linear and in the last row */
+	r = R(RD, 1, obj, 0, 1);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	r = R(WR, 2944 - 720 * 4 + 1, 720 * 4, 2944, 480);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	/* one row too many */
+	r = R(WR, 0, 720 * 4, 2944, 481);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	/* overflow in offset + size, and in rows * stride */
+	r = R(RD, U64_MAX - 7, 16, 0, 1);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	r = R(RD, 0, U64_MAX, 0, 1);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, obj), -EINVAL);
+	r = R(RD, U64_MAX - (u64)U32_MAX * 2, U32_MAX, U32_MAX, U32_MAX);
+	KUNIT_EXPECT_EQ(test, cache_range_check(&r, U64_MAX), -EINVAL);
+}
+
+/* The array checks come before any copy from userspace or handle lookup. */
+static void prime_test_range_array(struct kunit *test)
+{
+	struct drm_powervr_sgx_obj_array arr = {
+		.stride = sizeof(struct drm_powervr_sgx_cache_range),
+	};
+	struct pvrsgx_job job = {};
+
+	KUNIT_EXPECT_EQ(test, pvrsgx_prime_ranges(NULL, &arr, &job), 0);
+	KUNIT_EXPECT_EQ(test, job.nranges, 0);
+	arr.count = PVRSGX_MAX_CACHE_RANGES + 1;
+	KUNIT_EXPECT_EQ(test, pvrsgx_prime_ranges(NULL, &arr, &job), -EINVAL);
+	arr.count = 1;
+	arr.stride = 0;
+	KUNIT_EXPECT_EQ(test, pvrsgx_prime_ranges(NULL, &arr, &job), -EINVAL);
+	KUNIT_EXPECT_NULL(test, job.ranges);
+}
+
+/* An element longer than the kernel's with a non-zero tail: -E2BIG, before any lookup. */
+static void prime_test_range_tail(struct kunit *test)
+{
+	struct {
+		struct drm_powervr_sgx_cache_range r;
+		u64 tail;
+	} big = { .r = R(RD, 0, 64, 0, 1), .tail = 1 };
+	struct drm_powervr_sgx_obj_array arr = { .stride = sizeof(big), .count = 1 };
+	struct pvrsgx_job job = {};
+	unsigned long user;
+
+	user = kunit_vm_mmap(test, NULL, 0, PAGE_SIZE, PROT_READ | PROT_WRITE,
+			     MAP_ANONYMOUS | MAP_PRIVATE, 0);
+	KUNIT_ASSERT_FALSE(test, IS_ERR_VALUE(user));
+	KUNIT_ASSERT_NE(test, user, 0);
+	KUNIT_ASSERT_EQ(test, copy_to_user((void __user *)user, &big, sizeof(big)), 0);
+	arr.array = user;
+	KUNIT_EXPECT_EQ(test, pvrsgx_prime_ranges(NULL, &arr, &job), -E2BIG);
+	KUNIT_EXPECT_EQ(test, job.nranges, 0);
+}
+
+static struct kunit_case pvrsgx_prime_test_cases[] = {
+	KUNIT_CASE(prime_test_range_array),
+	KUNIT_CASE(prime_test_range_tail),
+	KUNIT_CASE(prime_test_range_accepts),
+	KUNIT_CASE(prime_test_range_refuses),
+	{}
+};
+
+static struct kunit_suite pvrsgx_prime_test_suite = {
+	.name = "powervr_sgx_prime",
+	.test_cases = pvrsgx_prime_test_cases,
+};
+
+kunit_test_suite(pvrsgx_prime_test_suite);
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_rt_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_rt_test.c
new file mode 100644
index 000000000..ea4e8a124
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_rt_test.c
@@ -0,0 +1,63 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/*
+ * KUnit tests of a render target's tiling numbers, included from
+ * pvrsgx_render.c: 720x480 at 1 and 4 samples, against the numbers that
+ * render that window correctly.
+ */
+
+#include <kunit/test.h>
+
+static void rt_test_720x480_1x(struct kunit *test)
+{
+	struct pvrsgx_rt_numbers n;
+
+	rt_numbers(&n, 720, 480, 1);
+	KUNIT_EXPECT_EQ(test, n.te_aa, 0x20000000);
+	KUNIT_EXPECT_EQ(test, n.mtile1, 0x82010018);
+	KUNIT_EXPECT_EQ(test, n.mtile2, 0x02010018);
+	KUNIT_EXPECT_EQ(test, n.te_screen, 0x0001d016);
+	KUNIT_EXPECT_EQ(test, n.mt_tiles, 0x40);
+	KUNIT_EXPECT_EQ(test, n.rt_flags, 0x88);
+	KUNIT_EXPECT_EQ(test, n.padx, 0x20);
+	KUNIT_EXPECT_EQ(test, n.rt_mt_tiles >> 4, 4);
+	KUNIT_EXPECT_EQ(test, n.rt_mt_tiles * 0xc, 0x300);
+	KUNIT_EXPECT_EQ(test, n.region_bytes, 0x3000);
+	KUNIT_EXPECT_EQ(test, n.tailptr_bytes, 0x1000);
+	/* ZLS control 0x001710b4 with the client's bits 0x00171004 */
+	KUNIT_EXPECT_EQ(test, (n.zls_xt - 1) << 3, 0xb0);
+}
+
+static void rt_test_720x480_4x(struct kunit *test)
+{
+	struct pvrsgx_rt_numbers n;
+
+	rt_numbers(&n, 720, 480, 4);
+	KUNIT_EXPECT_EQ(test, n.te_aa, 0xe0000000);
+	KUNIT_EXPECT_EQ(test, n.mtile1, 0x8180c012);
+	KUNIT_EXPECT_EQ(test, n.mtile2, 0x02010018);
+	KUNIT_EXPECT_EQ(test, n.te_screen, 0x0001d016);
+	KUNIT_EXPECT_EQ(test, n.mt_tiles, 0x30);
+	KUNIT_EXPECT_EQ(test, n.rt_flags, 0xeaa26e26);
+	KUNIT_EXPECT_EQ(test, n.padx, 0x30);
+	KUNIT_EXPECT_EQ(test, n.rt_mt_tiles >> 4, 0xc);
+	KUNIT_EXPECT_EQ(test, n.rt_mt_tiles * 0xc, 0x900);
+	KUNIT_EXPECT_EQ(test, n.region_bytes, 0x9000);
+	KUNIT_EXPECT_EQ(test, n.tailptr_bytes, 0x4000);
+	/* ZLS control 0x0017116c */
+	KUNIT_EXPECT_EQ(test, (n.zls_xt - 1) << 3, 0x168);
+}
+
+static struct kunit_case pvrsgx_rt_test_cases[] = {
+	KUNIT_CASE(rt_test_720x480_1x),
+	KUNIT_CASE(rt_test_720x480_4x),
+	{}
+};
+
+static struct kunit_suite pvrsgx_rt_test_suite = {
+	.name = "powervr_sgx_rt",
+	.test_cases = pvrsgx_rt_test_cases,
+};
+
+kunit_test_suite(pvrsgx_rt_test_suite);
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_usse_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_usse_test.c
new file mode 100644
index 000000000..0ef4a9d52
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_usse_test.c
@@ -0,0 +1,145 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/* KUnit tests of the USSE instruction check, included from pvrsgx_usse.c. */
+
+#include <kunit/test.h>
+
+#define W(hi, lo)	((u64)(hi) << 32 | (lo))
+
+/* Each word, alone at instruction 2 of a 512-instruction object placed at 0x100. */
+static const struct {
+	u64 w;
+	const char *what;
+	bool ok;
+} validate_words[] = {
+	{ W(0xfe210000, 0x2000001c), "ldr r0, #28", false },
+	{ W(0xfe290000, 0x20064080), "str #3200, r1", false },
+	{ W(0xf80000c0, 0x00000180), "setl r3", false },
+	{ W(0xf8000100, 0x00a00000), "savl r5", false },
+	{ W(0xf8000000, 0x00001077), "ba #4215", false },
+	{ W(0xf8000080, 0x00000000), "lapc", false },
+	{ W(0xf8000240, 0x00000003), "br #3 [link]", false },
+	{ W(0xf8000040, 0x0003fffe), "br #-2", true },
+	{ W(0xf8000040, 0x0003fffd), "br #-3", false },
+	{ W(0xf8000040, 0x000001fd), "br #509", true },
+	{ W(0xf8000040, 0x000001fe), "br #510", false },
+	{ W(0xfa440700, 0x00000000), "phas.i #0, #0 [wait=7]", true },
+	{ W(0xfa440000, 0x00000102), "phas.i #0x102 (in the object)", true },
+	{ W(0xfa440000, 0x000000ff), "phas.i #0xff (before it)", false },
+	{ W(0xfa440000, 0x00000300), "phas.i #0x300 (past it)", false },
+	{ W(0xfd200000, 0x00000010), "lock #1", false },
+	{ W(0xfb100010, 0x02003004), "smbo #1, #2, #3, #4", false },
+	{ W(0xdc2c23c5, 0xa0008001), "fdual.fssq.mov", false },
+	{ W(0xff200000, 0x00000001), "op0xd6 #1", false },
+	{ W(0xff200000, 0x00000000), "op0xd6 #0", true },
+	{ W(0x00080001, 0x08a04103), "fmad g5, r1, r2, r3", false },
+	{ W(0x28090001, 0x20600000), "mov c3, r0", false },
+	{ W(0x28030000, 0x60003900), "mov r0, g50", false },
+	{ W(0x28030000, 0x60002a00), "mov r0, g20", true },
+	{ W(0x28030000, 0xe0000200), "mov r0, i4", false },
+	{ W(0x28090003, 0x20800000), "mov i4, r0", false },
+	{ W(0x28a33001, 0x20003000), "mov with an indexed source", true },
+	{ W(0x00000000, 0xffffffff), "fmad with every operand bit set", true },
+	{ W(0x00000000, 0x00000000), "zero", true },
+	{ W(0xffffffff, 0xffffffff), "all ones", false },
+	/* Number bit 6: a flag only where the format makes it one. */
+	{ W(0x50021000, 0xc0000001), "and r0, i0, r1", true },
+	{ W(0x50021000, 0xc0002001), "and r0, i64, r1", false },
+	{ W(0x00420000, 0xc0006103), "fmad r0, r1, i2.f16, r3 [fmt]", true },
+	{ W(0x00020000, 0xc0006103), "fmad r0, r1, i66, r3", false },
+	{ W(0x9183f802, 0xf0002041), "sopwm pa0, i0.c10, i1.c10", true },
+	{ W(0xfd100000, 0x00000100), "setfc #0, #1", true },
+	{ W(0xfd100000, 0x00000101), "setfc #1, #1", false },
+};
+
+static void usse_test_words(struct kunit *test)
+{
+	for (int i = 0; i < ARRAY_SIZE(validate_words); i++) {
+		const u64 *w = &validate_words[i].w;
+		bool abs = false;
+		s64 bad = pvrsgx_usse_scan(PVRSGX_ISA_SGX545, w, 1, 2, 512, 0x100, &abs, NULL);
+
+		KUNIT_EXPECT_EQ_MSG(test, bad, validate_words[i].ok ? -1 : 0, "%s",
+				    validate_words[i].what);
+	}
+}
+
+/* phas.i with a target needs a place, and says it had one. */
+static void usse_test_phas_place(struct kunit *test)
+{
+	u64 w = W(0xfa440000, 0x00000102);
+	enum pvrsgx_usse_refusal why;
+	bool abs = false;
+
+	KUNIT_EXPECT_EQ(test, pvrsgx_usse_scan(PVRSGX_ISA_SGX545, &w, 1, 0, 512,
+					       PVRSGX_USSE_PLACE_NONE, &abs, &why), 0);
+	KUNIT_EXPECT_EQ(test, why, PVRSGX_USSE_REFUSAL_PHAS);
+	KUNIT_EXPECT_EQ(test, pvrsgx_usse_scan(PVRSGX_ISA_SGX545, &w, 1, 0, 512, 0x100, &abs,
+					       &why), -1);
+	KUNIT_EXPECT_EQ(test, why, PVRSGX_USSE_REFUSAL_NONE);
+	KUNIT_EXPECT_TRUE(test, abs);
+
+	/* An object with a target must not cross into the next window. */
+	w = W(0xfa440000, PVRSGX_USSE_WINDOW_INSTS - 0x100 + 2);
+	KUNIT_EXPECT_EQ(test, pvrsgx_usse_scan(PVRSGX_ISA_SGX545, &w, 1, 0, 0x100,
+					       PVRSGX_USSE_WINDOW_INSTS - 0x100, &abs, &why), -1);
+	KUNIT_EXPECT_EQ(test, pvrsgx_usse_scan(PVRSGX_ISA_SGX545, &w, 1, 0, 0x101,
+					       PVRSGX_USSE_WINDOW_INSTS - 0x100, &abs, &why), 0);
+	KUNIT_EXPECT_EQ(test, why, PVRSGX_USSE_REFUSAL_PHAS);
+}
+
+/* Each refusal says why. */
+static void usse_test_reasons(struct kunit *test)
+{
+	static const struct {
+		u64 w;
+		enum pvrsgx_usse_refusal why;
+	} cases[] = {
+		{ W(0xffffffff, 0xffffffff), PVRSGX_USSE_REFUSAL_FORM },
+		{ W(0x28030000, 0x60003900), PVRSGX_USSE_REFUSAL_OPERAND },	/* mov r0, g50 */
+		{ W(0xfe210000, 0x2000001c), PVRSGX_USSE_REFUSAL_KIND },	/* ldr */
+		{ W(0xf8000040, 0x000001fe), PVRSGX_USSE_REFUSAL_BRANCH },	/* br #510 */
+		{ W(0xfa440000, 0x000000ff), PVRSGX_USSE_REFUSAL_PHAS },	/* phas.i #0xff */
+	};
+
+	for (int i = 0; i < ARRAY_SIZE(cases); i++) {
+		enum pvrsgx_usse_refusal why;
+		bool abs = false;
+
+		KUNIT_EXPECT_EQ(test, pvrsgx_usse_scan(PVRSGX_ISA_SGX545, &cases[i].w, 1, 2, 512,
+						       0x100, &abs, &why), 0);
+		KUNIT_EXPECT_EQ(test, why, cases[i].why);
+	}
+}
+
+/* Every slot set holds slots the checker knows, with pieces that fit a word. */
+static void usse_test_table(struct kunit *test)
+{
+	for (int i = 0; i < ARRAY_SIZE(pvrsgx_usse_forms); i++)
+		KUNIT_EXPECT_LT(test, pvrsgx_usse_forms[i].slots, ARRAY_SIZE(pvrsgx_usse_slots));
+	for (int i = 0; i < ARRAY_SIZE(pvrsgx_usse_slots); i++) {
+		for (int k = 0; k < USSE_MAX_SLOTS; k++) {
+			const struct pvrsgx_usse_slot *s = &pvrsgx_usse_slots[i][k];
+
+			KUNIT_EXPECT_LE(test, s->kind, USSE_SLOT_SRC0);
+			for (int p = 0; p < USSE_MAX_PIECES; p++)
+				KUNIT_EXPECT_LE(test, s->p[p].lo + s->p[p].width, 64);
+		}
+	}
+}
+
+static struct kunit_case pvrsgx_usse_test_cases[] = {
+	KUNIT_CASE(usse_test_words),
+	KUNIT_CASE(usse_test_phas_place),
+	KUNIT_CASE(usse_test_reasons),
+	KUNIT_CASE(usse_test_table),
+	{}
+};
+
+static struct kunit_suite pvrsgx_usse_test_suite = {
+	.name = "powervr_sgx_usse",
+	.test_cases = pvrsgx_usse_test_cases,
+};
+
+kunit_test_suite(pvrsgx_usse_test_suite);
diff --git a/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_validate_test.c b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_validate_test.c
new file mode 100644
index 000000000..0fc2f197d
--- /dev/null
+++ b/drivers/gpu/drm/powervr_sgx/tests/pvrsgx_validate_test.c
@@ -0,0 +1,32 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright © 2026 Joseph Telaak */
+
+/* KUnit tests of code placement, included from pvrsgx_validate.c. */
+
+#include <kunit/test.h>
+
+/* Where a mapping puts a code object, in instructions from its heap's base. */
+static void validate_test_code_place(struct kunit *test)
+{
+	const struct pvrsgx_heap_layout *h = pvrsgx_core_family_get(545)->heaps;
+
+	KUNIT_EXPECT_EQ(test, code_place(h, PVRSGX_HEAP_USSE_PIXEL + 0x800, 0), 0x100);
+	KUNIT_EXPECT_EQ(test, code_place(h, PVRSGX_HEAP_USSE_VERTEX + 0x1000, 0x800), 0x100);
+	/* From the window, not the heap: window 1 starts at +512 KiB. */
+	KUNIT_EXPECT_EQ(test, code_place(h, PVRSGX_HEAP_USSE_PIXEL + 0x80800, 0), 0x100);
+	KUNIT_EXPECT_EQ(test, code_place(h, PVRSGX_HEAP_USSE_PIXEL + 0x80800, 0x1000), 0xff00);
+	KUNIT_EXPECT_EQ(test, code_place(h, PVRSGX_HEAP_USSE_PIXEL + 0x400, 0x800),
+			PVRSGX_USSE_PLACE_NONE);
+}
+
+static struct kunit_case pvrsgx_validate_test_cases[] = {
+	KUNIT_CASE(validate_test_code_place),
+	{}
+};
+
+static struct kunit_suite pvrsgx_validate_test_suite = {
+	.name = "powervr_sgx_validate",
+	.test_cases = pvrsgx_validate_test_cases,
+};
+
+kunit_test_suite(pvrsgx_validate_test_suite);

-- 
2.50.1

Re: [PATCH RFC 02/13] drm/gma500: Publish the Cedarview SGX as an auxiliary device

[Gabriel Gama]

Hi Joseph, I noticed that gma_sgx_set_irq_handler() updates sgx_data
before publishing the new handler pointer.
During unregister, could an in-flight IRQ still be using the previous
handler while sgx_data is being changed?
Would it make sense to clear the handler first, call
synchronize_irq(), and only then clear sgx_data?

Otherwise, the auxiliary-device approach looks interesting, especially
for keeping the display and render responsibilities separate

ty,
gabriel

Re: [PATCH RFC 02/13] drm/gma500: Publish the Cedarview SGX as an auxiliary device

[Joseph Telaak]

On Thu, Nov 12, 2026 at 11:08 PM Gabriel Gama wrote:
> Hi Joseph, I noticed that gma_sgx_set_irq_handler() updates sgx_data
> before publishing the new handler pointer.
> During unregister, could an in-flight IRQ still be using the previous
> handler while sgx_data is being changed?
> Would it make sense to clear the handler first, call
> synchronize_irq(), and only then clear sgx_data?

Yes, good catch. On unregister the data was cleared before the handler,
so a handler already running on another CPU could see NULL. For v2 it
does what you describe: the handler is cleared first, then
synchronize_irq(), and only then sgx_data. On register, sgx_data is set
before the handler is published, with a release/acquire pair against
the IRQ handler.

> Otherwise, the auxiliary-device approach looks interesting, especially
> for keeping the display and render responsibilities separate

Thanks, that was the main reason for it: gma500 keeps the display, and
the GPU driver doesn't need to know anything about it. The same SGX
cores sit next to quite different display hardware (gma500 on
Cedarview, simpledrm on the CE5300, omapdrm on OMAP3), so keeping the
render driver independent of the display side lets one driver cover
all of them.

Joseph

Re: [PATCH RFC 12/13] drm/powervr_sgx: Add the DRM driver, power management and debugging

[Uwe Kleine-König]

[-- Attachment #1: Type: text/plain, Size: 1581 bytes --]

On Thu, Nov 12, 2026 at 05:59:02PM -0500, Joseph Telaak wrote:
> +static const struct pci_device_id pvrsgx_pci_ids[] = {
> +	{ PCI_VDEVICE(INTEL, 0x089b), 545 },	/* CE5300 family: SGX545 */

Please make this

	{ PCI_VDEVICE(INTEL, 0x089b), .driver_data = 545 },	/* CE5300 family: SGX545 */

> +	{ }
> +};
> +MODULE_DEVICE_TABLE(pci, pvrsgx_pci_ids);
> [...]
> diff --git a/drivers/gpu/drm/powervr_sgx/pvrsgx_of.c b/drivers/gpu/drm/powervr_sgx/pvrsgx_of.c
> new file mode 100644
> index 000000000..e347a3699
> --- /dev/null
> +++ b/drivers/gpu/drm/powervr_sgx/pvrsgx_of.c
> @@ -0,0 +1,119 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright © 2026 Joseph Telaak */
> +
> +/*
> + * An SGX on a device tree SoC (Documentation/devicetree/bindings/gpu/
> + * img,powervr-sgx.yaml). On TI parts the GPU node has no clocks or power
> + * domain of its own: its ti-sysc parent module enables both, and applies the
> + * SGX interrupt quirk, when the GPU device is runtime-resumed. Every register
> + * access happens with the parent held awake.
> + */
> +
> +#include <linux/dma-mapping.h>
> +#include <linux/io.h>
> +#include <linux/mod_devicetable.h>

Don't include <linux/mod_devicetable.h>. of_device_id is already
provided by <linux/of.h> and also by <linux/platform_device.h> without
the useless dependencies that <linux/mod_devicetable.h> introduces.

> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/property.h>

Best regards
Uwe

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 12/13] drm/powervr_sgx: Add the DRM driver, power management and debugging

[Joseph Telaak]

On Fri, Nov 13, 2026 at 07:23:36AM +0100, Uwe Kleine-König wrote:
> On Thu, Nov 12, 2026 at 05:59:02PM -0500, Joseph Telaak wrote:
> > +static const struct pci_device_id pvrsgx_pci_ids[] = {
> > +     { PCI_VDEVICE(INTEL, 0x089b), 545 },    /* CE5300 family: SGX545 */
>
> Please make this
>
>         { PCI_VDEVICE(INTEL, 0x089b), .driver_data = 545 },     /* CE5300 family: SGX545 */

Will do in v2.

> Don't include <linux/mod_devicetable.h>. of_device_id is already
> provided by <linux/of.h> and also by <linux/platform_device.h> without
> the useless dependencies that <linux/mod_devicetable.h> introduces.

Thanks, I'll drop it in v2, and check the other files for the same.

Joseph