[PATCH v2 0/4] Add Toshiba Visconti AFFINE image processing accelerator driver

[PATCH v2 0/4] Add Toshiba Visconti AFFINE image processing accelerator driver

[Yuji Ishikawa]

This series is the AFFINE image processing accelerator driver for Toshiba's ARM SoC, Visconti[0].
This provides DT binding documentation, device driver, MAINTAINER files.

The second patch "soc: visconti: Add Toshiba Visconti image processing accelerator common source"
is commonly used among acclerator drivers (affine, dnn, dspif, pyramid).

Best regards,
Yuji

[0]: https://toshiba.semicon-storage.com/ap-en/semiconductor/product/image-recognition-processors-visconti.html
  
  dt-bindings: soc: visconti: Add Toshiba Visconti AFFINE image
    v1 -> v2:
      - No update

  soc: visconti: Add Toshiba Visconti image processing accelerator common source
    v1 -> v2:
      - apply checkpatch.pl --strict
  
  soc: visconti: Add Toshiba Visconti AFFINE image processing accelerator
    v1 -> v2:
      - apply checkpatch.pl --strict
      - rename hwd_AFFINE_xxxx to hwd_affine_xxxx

  MAINTAINERS: Add entries for Toshiba Visconti AFFINE image processing accelerator
    v1 -> v2:
      - No update

Change in V2:
  - apply checkpatch.pl --strict
  - rename hwd_AFFINE_xxxx to hwd_affine_xxxx

Yuji Ishikawa (4):
  dt-bindings: soc: visconti: Add Toshiba Visconti AFFINE image
    processing accelerator bindings
  soc: visconti: Add Toshiba Visconti image processing accelerator
    common source
  soc: visconti: Add Toshiba Visconti AFFINE image processing
    accelerator
  MAINTAINERS: Add entries for Toshiba Visconti AFFINE image processing
    accelerator

 .../soc/visconti/toshiba,visconti-affine.yaml |  53 ++
 MAINTAINERS                                   |   2 +
 drivers/soc/Kconfig                           |   1 +
 drivers/soc/Makefile                          |   1 +
 drivers/soc/visconti/Kconfig                  |   7 +
 drivers/soc/visconti/Makefile                 |   8 +
 drivers/soc/visconti/affine/Makefile          |   6 +
 drivers/soc/visconti/affine/affine.c          | 451 ++++++++++++++++++
 drivers/soc/visconti/affine/hwd_affine.c      | 206 ++++++++
 drivers/soc/visconti/affine/hwd_affine.h      |  83 ++++
 drivers/soc/visconti/affine/hwd_affine_reg.h  |  45 ++
 drivers/soc/visconti/ipa_common.c             |  55 +++
 drivers/soc/visconti/ipa_common.h             |  18 +
 drivers/soc/visconti/uapi/affine.h            |  87 ++++
 drivers/soc/visconti/uapi/ipa.h               |  88 ++++
 15 files changed, 1111 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/soc/visconti/toshiba,visconti-affine.yaml
 create mode 100644 drivers/soc/visconti/Kconfig
 create mode 100644 drivers/soc/visconti/Makefile
 create mode 100644 drivers/soc/visconti/affine/Makefile
 create mode 100644 drivers/soc/visconti/affine/affine.c
 create mode 100644 drivers/soc/visconti/affine/hwd_affine.c
 create mode 100644 drivers/soc/visconti/affine/hwd_affine.h
 create mode 100644 drivers/soc/visconti/affine/hwd_affine_reg.h
 create mode 100644 drivers/soc/visconti/ipa_common.c
 create mode 100644 drivers/soc/visconti/ipa_common.h
 create mode 100644 drivers/soc/visconti/uapi/affine.h
 create mode 100644 drivers/soc/visconti/uapi/ipa.h

-- 
2.17.1

[PATCH v2 1/4] dt-bindings: soc: visconti: Add Toshiba Visconti AFFINE image processing accelerator bindings

[Yuji Ishikawa]

Adds the Device Tree binding documentation that allows to describe
the AFFINE image processing accelerator found in Toshiba Visconti SoCs.

Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
 .../soc/visconti/toshiba,visconti-affine.yaml | 53 +++++++++++++++++++
 1 file changed, 53 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/soc/visconti/toshiba,visconti-affine.yaml

diff --git a/Documentation/devicetree/bindings/soc/visconti/toshiba,visconti-affine.yaml b/Documentation/devicetree/bindings/soc/visconti/toshiba,visconti-affine.yaml
new file mode 100644
index 000000000..a446e1c4f
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/visconti/toshiba,visconti-affine.yaml
@@ -0,0 +1,53 @@
+# SPDX-LIcense-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soc/visconti/toshiba,visconti-affine.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Toshiba Visconti AFFINE image processing accelerator
+
+maintainers:
+  - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
+
+description: |
+  Toshiba Visconti AFFINE image processing accelerator provides affine transform, lens undistortion and LUT transform.
+  Visconti5 have up to 2 AFFINE units.
+
+properties:
+  compatible:
+    items:
+      - const: toshiba,visconti-affine
+
+  reg:
+    maxItems: 1
+
+  interrupts:
+    maxItems: 1
+
+  index:
+    enum: [0, 1]
+
+required:
+  - compatible
+  - reg
+  - interrupts
+  - index
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/irq.h>
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+        affine0: affine@14000000 {
+            compatible = "toshiba,visconti-affine";
+            reg = <0 0x14000000 0 0x8000>;
+            interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_HIGH>;
+            index = <0>;
+            status = "disabled";
+        };
+    };
-- 
2.17.1

[PATCH v2 2/4] soc: visconti: Add Toshiba Visconti image processing accelerator common source

[Yuji Ishikawa]

Adds common operations for image processing accelerator drivers
including dma-buf control and ioctl definitiion

Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
v1 -> v2:
  - apply checkpatch.pl --strict
---
 drivers/soc/Kconfig               |  1 +
 drivers/soc/Makefile              |  1 +
 drivers/soc/visconti/Kconfig      |  1 +
 drivers/soc/visconti/Makefile     |  6 +++
 drivers/soc/visconti/ipa_common.c | 55 +++++++++++++++++++
 drivers/soc/visconti/ipa_common.h | 18 +++++++
 drivers/soc/visconti/uapi/ipa.h   | 88 +++++++++++++++++++++++++++++++
 7 files changed, 170 insertions(+)
 create mode 100644 drivers/soc/visconti/Kconfig
 create mode 100644 drivers/soc/visconti/Makefile
 create mode 100644 drivers/soc/visconti/ipa_common.c
 create mode 100644 drivers/soc/visconti/ipa_common.h
 create mode 100644 drivers/soc/visconti/uapi/ipa.h

diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig
index e8a30c4c5..c99139aa8 100644
--- a/drivers/soc/Kconfig
+++ b/drivers/soc/Kconfig
@@ -22,6 +22,7 @@ source "drivers/soc/tegra/Kconfig"
 source "drivers/soc/ti/Kconfig"
 source "drivers/soc/ux500/Kconfig"
 source "drivers/soc/versatile/Kconfig"
+source "drivers/soc/visconti/Kconfig"
 source "drivers/soc/xilinx/Kconfig"
 
 endmenu
diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile
index a05e9fbcd..455b993c2 100644
--- a/drivers/soc/Makefile
+++ b/drivers/soc/Makefile
@@ -28,4 +28,5 @@ obj-$(CONFIG_ARCH_TEGRA)	+= tegra/
 obj-y				+= ti/
 obj-$(CONFIG_ARCH_U8500)	+= ux500/
 obj-$(CONFIG_PLAT_VERSATILE)	+= versatile/
+obj-$(CONFIG_ARCH_VISCONTI)	+= visconti/
 obj-y				+= xilinx/
diff --git a/drivers/soc/visconti/Kconfig b/drivers/soc/visconti/Kconfig
new file mode 100644
index 000000000..8b1378917
--- /dev/null
+++ b/drivers/soc/visconti/Kconfig
@@ -0,0 +1 @@
+
diff --git a/drivers/soc/visconti/Makefile b/drivers/soc/visconti/Makefile
new file mode 100644
index 000000000..8d710da08
--- /dev/null
+++ b/drivers/soc/visconti/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Visconti specific device drivers.
+#
+
+obj-y += ipa_common.o
diff --git a/drivers/soc/visconti/ipa_common.c b/drivers/soc/visconti/ipa_common.c
new file mode 100644
index 000000000..6345f33c5
--- /dev/null
+++ b/drivers/soc/visconti/ipa_common.c
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/* Toshiba Visconti Image Processing Accelerator Support
+ *
+ * (C) Copyright 2022 TOSHIBA CORPORATION
+ * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#include "ipa_common.h"
+
+int ipa_attach_dmabuf(struct device *dev, int fd, struct dma_buf_attachment **a,
+		      struct sg_table **s, dma_addr_t *addr, enum dma_data_direction dma_dir)
+{
+	struct dma_buf_attachment *attachment;
+	struct dma_buf *dmabuf;
+	struct sg_table *sgt;
+	int ret;
+
+	dmabuf = dma_buf_get(fd);
+	if (IS_ERR(dmabuf)) {
+		dev_err(dev, "Invalid dmabuf FD\n");
+		return PTR_ERR(dmabuf);
+	}
+	attachment = dma_buf_attach(dmabuf, dev);
+
+	if (IS_ERR(attachment)) {
+		dev_err(dev, "Failed to attach dmabuf\n");
+		ret = PTR_ERR(attachment);
+		goto err_put;
+	}
+	sgt = dma_buf_map_attachment(attachment, dma_dir);
+	if (IS_ERR(sgt)) {
+		dev_err(dev, "Failed to get dmabufs sg_table\n");
+		ret = PTR_ERR(sgt);
+		goto err_detach;
+	}
+	if (sgt->nents != 1) {
+		dev_err(dev, "Sparse DMA region is unsupported\n");
+		ret = -EINVAL;
+		goto err_unmap;
+	}
+
+	*addr = sg_dma_address(sgt->sgl);
+	*a = attachment;
+	*s = sgt;
+
+	return 0;
+
+err_unmap:
+	dma_buf_unmap_attachment(attachment, sgt, dma_dir);
+err_detach:
+	dma_buf_detach(dmabuf, attachment);
+err_put:
+	dma_buf_put(dmabuf);
+	return ret;
+}
diff --git a/drivers/soc/visconti/ipa_common.h b/drivers/soc/visconti/ipa_common.h
new file mode 100644
index 000000000..786988c52
--- /dev/null
+++ b/drivers/soc/visconti/ipa_common.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/* Toshiba Visconti Image Processing Accelerator Support
+ *
+ * (C) Copyright 2022 TOSHIBA CORPORATION
+ * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#include <linux/dma-mapping.h>
+#include <linux/dma-buf.h>
+
+#define COHERENT_ADDRESS_BIT   (0x400000000ULL)
+#define IPA_POLL_EVENT_NONE    (0)
+#define IPA_POLL_EVENT_DONE    (1)
+#define IPA_POLL_EVENT_ERROR   (2)
+#define IPA_WAKEUP_RETRY_DELAY (300 * 1000) /*usec*/
+
+int ipa_attach_dmabuf(struct device *dev, int fd, struct dma_buf_attachment **a,
+		      struct sg_table **s, dma_addr_t *addr, enum dma_data_direction dma_dir);
diff --git a/drivers/soc/visconti/uapi/ipa.h b/drivers/soc/visconti/uapi/ipa.h
new file mode 100644
index 000000000..e5c919ea2
--- /dev/null
+++ b/drivers/soc/visconti/uapi/ipa.h
@@ -0,0 +1,88 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* Toshiba Visconti Image Processing Accelerator Support
+ *
+ * (C) Copyright 2022 TOSHIBA CORPORATION
+ * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#ifndef _UAPI_LINUX_IPA_H
+#define _UAPI_LINUX_IPA_H
+
+/**
+ * enum drv_ipa_state - state of Visconti IPA driver
+ *
+ * @DRV_IPA_STATE_IDLE:     driver is idle
+ * @DRV_IPA_STATE_BUSY:     device is busy
+ */
+enum drv_ipa_state { DRV_IPA_STATE_IDLE = 0, DRV_IPA_STATE_BUSY };
+
+/**
+ * enum drv_ipa_buffer_direction - usage of buffer
+ *
+ * @DRV_IPA_DIR_BIDIRECTION: cpu writes/reads data
+ * @DRV_IPA_DIR_TO_DEVICE:   cpu writes data
+ * @DRV_IPA_DIR_FROM_DEVICE: cpu reads data
+ * @DRV_IPA_DIR_NONE:        cpu not access or non-coherent
+ */
+enum drv_ipa_buffer_direction {
+	DRV_IPA_DIR_BIDIRECTION = 0,
+	DRV_IPA_DIR_TO_DEVICE,
+	DRV_IPA_DIR_FROM_DEVICE,
+	DRV_IPA_DIR_NONE
+};
+
+/**
+ * struct drv_ipa_buffer_info - buffer information for Visconti IPA drivers
+ *
+ * @fd:        file descriptor of Ion allocated heap
+ * @coherent:  allocated via coherent bus or not
+ * @direction: buffer direction (to/from device)
+ *
+ * note: When the Ion heap is allocated wit ION_FLAG_CACHED,
+ * &struct drv_ipa_buffer_info.coherent should be true.
+ * Else, it should be false.
+ */
+struct drv_ipa_buffer_info {
+	u32 fd;
+	bool coherent;
+	enum drv_ipa_buffer_direction direction;
+};
+
+/**
+ * struct drv_ipa_addr - address structure for Visconti IPA drivers
+ *
+ * @buffer_index: array index of &struct drv_ipa_buffer_info
+ * @offset:       offset from the top of Ion heap
+ */
+struct drv_ipa_addr {
+	u32 buffer_index : 4;
+	u32 offset : 28;
+};
+
+#define IPA_BUFFER_INDEX_MAX (16)
+#define IPA_INVALID_ADDR                                                                           \
+	{                                                                                          \
+		.buffer_index = 0xfu, .offset = 0xfffffffu                                         \
+	}
+
+static inline struct drv_ipa_addr drv_ipa_invalid_addr(void)
+{
+	struct drv_ipa_addr invalid_addr = IPA_INVALID_ADDR;
+	return invalid_addr;
+}
+
+static inline bool drv_ipa_is_invalid_addr(struct drv_ipa_addr addr)
+{
+	struct drv_ipa_addr invalid_addr = IPA_INVALID_ADDR;
+
+	if (addr.buffer_index == invalid_addr.buffer_index && addr.offset == invalid_addr.offset) {
+		return true;
+	}
+	return false;
+}
+
+#define IOC_IPA_MAGIC	   'I'
+#define IOC_IPA_START	   _IO(IOC_IPA_MAGIC, 0)
+#define IOC_IPA_GET_STATUS _IO(IOC_IPA_MAGIC, 1)
+
+#endif /* _UAPI_LINUX_IPA_H */
-- 
2.17.1

[PATCH v2 3/4] soc: visconti: Add Toshiba Visconti AFFINE image processing accelerator

[Yuji Ishikawa]

Adds support to AFFINE image processing accelerator on Toshiba Visconti ARM SoCs.
This accelerator supoorts affine transform, lens undistortion and LUT transform.

Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
v1 -> v2:
  - apply checkpatch.pl --strict
  - renamed identifiers; hwd_AFFINE_xxxx to hwd_affine_xxxx
---
 drivers/soc/visconti/Kconfig                 |   6 +
 drivers/soc/visconti/Makefile                |   2 +
 drivers/soc/visconti/affine/Makefile         |   6 +
 drivers/soc/visconti/affine/affine.c         | 451 +++++++++++++++++++
 drivers/soc/visconti/affine/hwd_affine.c     | 206 +++++++++
 drivers/soc/visconti/affine/hwd_affine.h     |  83 ++++
 drivers/soc/visconti/affine/hwd_affine_reg.h |  45 ++
 drivers/soc/visconti/uapi/affine.h           |  87 ++++
 8 files changed, 886 insertions(+)
 create mode 100644 drivers/soc/visconti/affine/Makefile
 create mode 100644 drivers/soc/visconti/affine/affine.c
 create mode 100644 drivers/soc/visconti/affine/hwd_affine.c
 create mode 100644 drivers/soc/visconti/affine/hwd_affine.h
 create mode 100644 drivers/soc/visconti/affine/hwd_affine_reg.h
 create mode 100644 drivers/soc/visconti/uapi/affine.h

diff --git a/drivers/soc/visconti/Kconfig b/drivers/soc/visconti/Kconfig
index 8b1378917..01583d407 100644
--- a/drivers/soc/visconti/Kconfig
+++ b/drivers/soc/visconti/Kconfig
@@ -1 +1,7 @@
+if ARCH_VISCONTI
+
+config VISCONTI_AFFINE
+    bool "Visconti Affine driver"
+
+endif
 
diff --git a/drivers/soc/visconti/Makefile b/drivers/soc/visconti/Makefile
index 8d710da08..b25a726c3 100644
--- a/drivers/soc/visconti/Makefile
+++ b/drivers/soc/visconti/Makefile
@@ -4,3 +4,5 @@
 #
 
 obj-y += ipa_common.o
+
+obj-$(CONFIG_VISCONTI_AFFINE) += affine/
diff --git a/drivers/soc/visconti/affine/Makefile b/drivers/soc/visconti/affine/Makefile
new file mode 100644
index 000000000..82f83b2d6
--- /dev/null
+++ b/drivers/soc/visconti/affine/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Visconti AFFINE driver
+#
+
+obj-y += affine.o hwd_affine.o
diff --git a/drivers/soc/visconti/affine/affine.c b/drivers/soc/visconti/affine/affine.c
new file mode 100644
index 000000000..eea045dcf
--- /dev/null
+++ b/drivers/soc/visconti/affine/affine.c
@@ -0,0 +1,451 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/* Toshiba Visconti Affine Accelerator Support
+ *
+ * (C) Copyright 2022 TOSHIBA CORPORATION
+ * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#include <linux/delay.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/iommu.h>
+#include <linux/kernel.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/poll.h>
+#include <linux/wait.h>
+
+#include "hwd_affine.h"
+#include "../ipa_common.h"
+#include "../uapi/affine.h"
+
+struct affine_priv {
+	struct device *dev;
+	struct miscdevice miscdev;
+	struct mutex lock;
+	void __iomem *regs;
+	int irq;
+	wait_queue_head_t waitq;
+	enum drv_ipa_state status;
+	unsigned int hwd_event;
+	unsigned int poll_event;
+	int id;
+	char name[16];
+	bool dma_coherent;
+	struct hwd_affine_status hwd_status;
+
+	struct dma_buf_attachment *dba[DRV_AFFINE_BUFFER_INDEX_MAX];
+	struct sg_table *sgt[DRV_AFFINE_BUFFER_INDEX_MAX];
+	enum dma_data_direction dma_dir[DRV_AFFINE_BUFFER_INDEX_MAX];
+	unsigned int dma_count;
+
+	dma_addr_t buffer_iova[DRV_AFFINE_BUFFER_INDEX_MAX];
+};
+
+static u32 affine_ipa_addr_to_iova(struct affine_priv *priv, struct drv_ipa_addr addr)
+{
+	u32 iova = 0;
+
+	if (addr.buffer_index < priv->dma_count &&
+	    addr.offset < priv->dba[addr.buffer_index]->dmabuf->size)
+		iova = priv->buffer_iova[addr.buffer_index] + addr.offset;
+	return iova;
+}
+
+static int affine_attach_dma_buf(struct affine_priv *priv, unsigned int buffer_index,
+				 struct drv_ipa_buffer_info *buffer_info)
+{
+	int ret = 0;
+	dma_addr_t addr;
+
+	if (buffer_index >= DRV_AFFINE_BUFFER_INDEX_MAX) {
+		dev_err(priv->dev, "Buffer index invalid: index=%d\n", buffer_index);
+		return -EINVAL;
+	}
+
+	switch (buffer_info[buffer_index].direction) {
+	case DRV_IPA_DIR_NONE:
+		priv->dma_dir[priv->dma_count] = DMA_NONE;
+		break;
+	case DRV_IPA_DIR_TO_DEVICE:
+		priv->dma_dir[priv->dma_count] = DMA_TO_DEVICE;
+		break;
+	case DRV_IPA_DIR_FROM_DEVICE:
+		priv->dma_dir[priv->dma_count] = DMA_FROM_DEVICE;
+		break;
+	case DRV_IPA_DIR_BIDIRECTION:
+		priv->dma_dir[priv->dma_count] = DMA_BIDIRECTIONAL;
+		break;
+	default:
+		dev_err(priv->dev, "DMA direction invalid: index=%d dir=%d\n", buffer_index,
+			buffer_info[buffer_index].direction);
+		return -EINVAL;
+	}
+
+	if (!buffer_info[buffer_index].coherent) {
+		priv->dev->dma_coherent = false;
+		if (priv->dma_coherent)
+			priv->dma_dir[priv->dma_count] = DMA_NONE;
+	}
+
+	ret = ipa_attach_dmabuf(priv->dev, buffer_info[buffer_index].fd,
+				&priv->dba[priv->dma_count], &priv->sgt[priv->dma_count], &addr,
+				priv->dma_dir[priv->dma_count]);
+	if (ret == 0) {
+		priv->dma_count++;
+		priv->buffer_iova[buffer_index] = addr;
+	}
+
+	priv->dev->dma_coherent = priv->dma_coherent;
+
+	return ret;
+}
+
+static void affine_detach_dma_buf(struct affine_priv *priv)
+{
+	struct dma_buf *dmabuf;
+	int i;
+
+	for (i = 0; i < priv->dma_count; i++) {
+		dmabuf = priv->dba[i]->dmabuf;
+		dma_buf_unmap_attachment(priv->dba[i], priv->sgt[i], priv->dma_dir[i]);
+		dma_buf_detach(dmabuf, priv->dba[i]);
+		dma_buf_put(dmabuf);
+	}
+}
+
+static irqreturn_t affine_irq(int irq, void *dev_id)
+{
+	struct affine_priv *priv = dev_id;
+
+	priv->hwd_event = hwd_affine_irq_handler(priv->id);
+
+	disable_irq_nosync(priv->irq);
+
+	return IRQ_WAKE_THREAD;
+}
+
+static irqreturn_t affine_irq_thread(int irq, void *dev_id)
+{
+	struct affine_priv *priv = dev_id;
+	unsigned long delay = 1;
+
+	mutex_lock(&priv->lock);
+	affine_detach_dma_buf(priv);
+
+	hwd_affine_get_status(priv->id, &priv->hwd_status);
+
+	priv->status = DRV_IPA_STATE_IDLE;
+
+	/* status should be updated before poll_event so that
+	 * when poll() returns, user context must observe state as idle
+	 */
+	smp_wmb();
+
+	if (priv->hwd_event == HWD_AFFINE_EVENT_DONE)
+		priv->poll_event = IPA_POLL_EVENT_DONE;
+	else
+		priv->poll_event = IPA_POLL_EVENT_ERROR;
+
+	/* General barrier to avoid re-ordering of priv->poll_event=N and
+	 * waitqueue_active()
+	 */
+	smp_mb();
+
+	/* Threads going to sleep in poll() can miss wakeup, when wakeup is done
+	 * between event check in ipa_poll() and sleeping. Wakeup repeatedly.
+	 */
+	while (waitqueue_active(&priv->waitq)) {
+		wake_up_interruptible(&priv->waitq);
+
+		WARN_ON(delay > IPA_WAKEUP_RETRY_DELAY);
+		usleep_range(delay, delay + 1);
+		delay += delay;
+	}
+
+	mutex_unlock(&priv->lock);
+
+	return IRQ_HANDLED;
+}
+
+static void affine_start(struct affine_priv *priv, struct hwd_affine_descriptor *desc)
+{
+	hwd_affine_start(priv->id, desc);
+}
+
+static int affine_ioctl_start(struct affine_priv *priv, unsigned long arg)
+{
+	struct hwd_affine_descriptor hwd_desc;
+	struct drv_affine_descriptor desc;
+	int ret = 0;
+	int i;
+
+	ret = mutex_lock_interruptible(&priv->lock);
+	if (ret)
+		return ret;
+
+	if (priv->status == DRV_IPA_STATE_BUSY) {
+		dev_dbg(priv->dev, "busy: %d\n", priv->status);
+		ret = -EBUSY;
+		goto err1;
+	}
+
+	if (copy_from_user(&desc, (void __user *)arg, sizeof(struct drv_affine_descriptor))) {
+		dev_err(priv->dev, "Descriptor memory access error\n");
+		ret = -EFAULT;
+		goto err1;
+	}
+
+	if (DRV_AFFINE_BIT_CONFIG_DESC_FINAL !=
+	    (desc.config_done & DRV_AFFINE_BIT_CONFIG_DESC_FINAL)) {
+		dev_err(priv->dev, "Descriptor configuration not complete\n");
+		ret = -EINVAL;
+		goto err1;
+	}
+
+	priv->dma_count = 0;
+
+	/* setup buffer */
+	for (i = 0; i < desc.buffer_info_num; i++) {
+		ret = affine_attach_dma_buf(priv, i, desc.buffer_info);
+		if (ret) {
+			dev_err(priv->dev, "dma buf attach error: index=%d\n", i);
+			goto err2;
+		}
+		dev_dbg(priv->dev, "@buffer[%d]@: fd=%d %s iova=%llx\n", i, desc.buffer_info[i].fd,
+			desc.buffer_info[i].coherent ? "coherent" : "non-coherent",
+			(uint64_t)priv->buffer_iova[i]);
+	}
+
+	memcpy(&hwd_desc, &desc, sizeof(struct hwd_affine_descriptor));
+
+	hwd_desc.src_addr = affine_ipa_addr_to_iova(priv, desc.src_addr);
+	if (hwd_desc.src_addr == 0) {
+		dev_err(priv->dev, "IPA address to iova conversion error: src_addr %s: %d\n",
+			__func__, __LINE__);
+		ret = -EINVAL;
+		goto err2;
+	}
+
+	hwd_desc.dst_addr = affine_ipa_addr_to_iova(priv, desc.dst_addr);
+	if (hwd_desc.dst_addr == 0) {
+		dev_err(priv->dev, "IPA address to iova conversion error: dst_addr %s: %d\n",
+			__func__, __LINE__);
+		ret = -EINVAL;
+		goto err2;
+	}
+
+	hwd_desc.tbl_addr = 0;
+	if (desc.tbl_ptch != 0) {
+		hwd_desc.tbl_addr = affine_ipa_addr_to_iova(priv, desc.tbl_addr);
+		if (hwd_desc.tbl_addr == 0) {
+			dev_err(priv->dev,
+				"IPA address to iova conversion error: tbl_addr %s: %d\n", __func__,
+				__LINE__);
+			ret = -EINVAL;
+			goto err2;
+		}
+	}
+
+	dev_dbg(priv->dev, "src: 0x%x\n", hwd_desc.src_addr);
+	dev_dbg(priv->dev, "dst: 0x%x\n", hwd_desc.dst_addr);
+	dev_dbg(priv->dev, "tbl: 0x%x\n", hwd_desc.tbl_addr);
+
+	affine_start(priv, &hwd_desc);
+
+	priv->poll_event = IPA_POLL_EVENT_NONE;
+	priv->hwd_event = 0;
+	priv->status = DRV_IPA_STATE_BUSY;
+	/* Barrier to prevent affine_irq() from setting priv->hwd_event
+	 * before it is reset above
+	 */
+	smp_wmb();
+	enable_irq(priv->irq);
+
+	mutex_unlock(&priv->lock);
+
+	return ret;
+
+err2:
+	affine_detach_dma_buf(priv);
+err1:
+	mutex_unlock(&priv->lock);
+	return ret;
+}
+
+static int affine_ioctl_get_status(struct affine_priv *priv, unsigned long arg)
+{
+	struct drv_affine_status status;
+	struct hwd_affine_status hwd_status;
+	int ret = 0;
+
+	ret = mutex_lock_interruptible(&priv->lock);
+	if (ret)
+		return ret;
+
+	if (priv->status == DRV_IPA_STATE_BUSY)
+		hwd_affine_get_status(priv->id, &hwd_status);
+	else
+		hwd_status = priv->hwd_status;
+
+	status.state = priv->status;
+	mutex_unlock(&priv->lock);
+
+	status.comp_lack_data = hwd_status.comp_lack_data;
+	status.comp_over_coordinate = hwd_status.comp_over_coordinate;
+	status.comp_over_data = hwd_status.comp_over_data;
+	status.comp_size = hwd_status.comp_size;
+	status.comp_syntax_error = hwd_status.comp_syntax_error;
+	if (copy_to_user((void __user *)arg, &status, sizeof(struct drv_affine_status))) {
+		dev_err(priv->dev, "status memory access error\n");
+		ret = -EFAULT;
+	}
+
+	return ret;
+}
+
+static long affine_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
+{
+	struct affine_priv *priv = container_of(fp->private_data, struct affine_priv, miscdev);
+	int ret = 0;
+
+	switch (cmd) {
+	case IOC_IPA_START:
+		ret = affine_ioctl_start(priv, arg);
+		break;
+	case IOC_IPA_GET_STATUS:
+		ret = affine_ioctl_get_status(priv, arg);
+		break;
+	default:
+		ret = -ENOIOCTLCMD;
+		break;
+	}
+
+	return ret;
+}
+
+static __poll_t affine_poll(struct file *fp, poll_table *wait)
+{
+	struct affine_priv *priv = container_of(fp->private_data, struct affine_priv, miscdev);
+	__poll_t mask = 0;
+	unsigned int poll_event;
+
+	poll_wait(fp, &priv->waitq, wait);
+
+	/* Barrier to avoid re-ordering of poll_wait() and event load
+	 * Read barrier here and release barrier in poll_wait() together will
+	 * prevent re-ordering
+	 */
+	smp_rmb();
+	poll_event = priv->poll_event;
+	if (poll_event != IPA_POLL_EVENT_NONE) {
+		if (poll_event == IPA_POLL_EVENT_DONE)
+			mask = EPOLLIN | EPOLLRDNORM;
+		else
+			mask = EPOLLERR;
+	}
+	return mask;
+}
+
+static const struct file_operations affine_fops = {
+	.owner = THIS_MODULE,
+	.unlocked_ioctl = affine_ioctl,
+	.poll = affine_poll,
+};
+
+static int affine_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct affine_priv *priv;
+	int ret;
+
+	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+	if (!priv)
+		return -ENOMEM;
+
+	mutex_init(&priv->lock);
+
+	/* update DMA mask */
+	priv->dma_coherent = dev->dma_coherent;
+	ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(36));
+	if (ret)
+		return ret;
+
+	priv->regs = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(priv->regs))
+		return PTR_ERR(priv->regs);
+
+	priv->irq = platform_get_irq(pdev, 0);
+	if (priv->irq < 0) {
+		dev_err(dev, "failed to acquire irq resource\n");
+		return -ENOENT;
+	}
+	ret = devm_request_threaded_irq(dev, priv->irq, affine_irq, affine_irq_thread, 0, "affine",
+					priv);
+	if (ret) {
+		dev_err(dev, "irq request failed\n");
+		return ret;
+	}
+	disable_irq(priv->irq);
+
+	ret = of_property_read_u32(dev->of_node, "index", &priv->id);
+	if (ret) {
+		dev_err(dev, "failed to acquire irq resource\n");
+		return ret;
+	}
+
+	hwd_affine_initialize(priv->id, priv->regs);
+
+	snprintf(priv->name, sizeof(priv->name), "affine%d", priv->id);
+	priv->miscdev.minor = MISC_DYNAMIC_MINOR;
+	priv->miscdev.name = priv->name;
+	priv->miscdev.fops = &affine_fops;
+	ret = misc_register(&priv->miscdev);
+	if (ret) {
+		dev_err(dev, "misc registration failed\n");
+		hwd_affine_uninitialize(priv->id);
+		return ret;
+	}
+
+	priv->dev = dev;
+	platform_set_drvdata(pdev, priv);
+
+	init_waitqueue_head(&priv->waitq);
+
+	priv->status = DRV_IPA_STATE_IDLE;
+	return 0;
+}
+
+static int affine_remove(struct platform_device *pdev)
+{
+	struct affine_priv *priv = platform_get_drvdata(pdev);
+
+	misc_deregister(&priv->miscdev);
+	hwd_affine_uninitialize(priv->id);
+	return 0;
+}
+
+static const struct of_device_id affine_of_match[] = {
+	{
+		.compatible = "toshiba,visconti-affine",
+	},
+	{},
+};
+MODULE_DEVICE_TABLE(platform, affine_of_match);
+
+static struct platform_driver affine_driver = {
+	.probe = affine_probe,
+	.remove = affine_remove,
+	.driver = {
+		.name = "visconti_affine",
+		.of_match_table = of_match_ptr(affine_of_match),
+	},
+};
+module_platform_driver(affine_driver);
+
+MODULE_AUTHOR("Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>");
+MODULE_DESCRIPTION("Toshiba Visconti affine driver");
+MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/soc/visconti/affine/hwd_affine.c b/drivers/soc/visconti/affine/hwd_affine.c
new file mode 100644
index 000000000..6720a0212
--- /dev/null
+++ b/drivers/soc/visconti/affine/hwd_affine.c
@@ -0,0 +1,206 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/* Toshiba Visconti Affine Accelerator Support
+ *
+ * (C) Copyright 2022 TOSHIBA CORPORATION
+ * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#include <linux/delay.h>
+#include <linux/io.h>
+
+#include "hwd_affine.h"
+#include "hwd_affine_reg.h"
+
+#define HWD_AFFINE_CLEAR_BITS	  GENMASK(15, 0)
+#define HWD_AFFINE_INTMASK_NORMAL (0x04100)
+
+#define HWD_AFFINE_INT_END	  BIT(0)
+#define HWD_AFFINE_INT_TCOMP_ERR0 BIT(16)
+#define HWD_AFFINE_INT_TCOMP_ERR1 BIT(17)
+#define HWD_AFFINE_INT_TCOMP_ERR2 BIT(18)
+#define HWD_AFFINE_INT_TCOMP_ERR3 BIT(19)
+#define HWD_AFFINE_INT_TCOMP_SIZE BIT(20)
+
+#define HWD_AFFINE_HWD_MASK_ADDR GENMASK(31, 0)
+
+#define HWD_AFFINE_START_CMD BIT(0)
+
+/**
+ * struct hwd_affine_resources - HWD driver internal resource structure
+ */
+struct hwd_affine_resources {
+	struct hwd_affine_reg *reg;
+	struct hwd_affine_status status;
+};
+
+/* HWD driver internal resource */
+static struct hwd_affine_resources hwd_affine_resources[HWD_AFFINE_DEVICE_MAX] = {};
+
+/**
+ * hwd_affine_initialize() - Initialize AFFINE device
+ *
+ * @module_id: @ref hwd_affine_device "id" of the h/w module
+ * @vaddr: register base virtual address
+ */
+void hwd_affine_initialize(u32 module_id, void *vaddr)
+{
+	struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
+
+	/* Initialize the device */
+	res->reg = (struct hwd_affine_reg *)vaddr;
+}
+
+/**
+ * hwd_affine_uninitialize() - Uninitialize AFFINE device
+ *
+ * @module_id: @ref hwd_affine_device "id" of the h/w module
+ */
+void hwd_affine_uninitialize(u32 module_id)
+{
+	struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
+
+	/* Deinitialize the device */
+	res->reg = NULL;
+}
+
+#define NUM_AFFINE_PARAMS  6
+#define NUM_HOMO_PARAMS	   3
+#define NUM_DISTORT_PARAMS 12
+
+/**
+ * hwd_affine_start() - Start AFFINE device
+ *
+ * @module_id: @ref hwd_affine_device "id" of the h/w module
+ * @desc: Pointer to AFFINE descriptor structure
+ */
+void hwd_affine_start(u32 module_id, const struct hwd_affine_descriptor *desc)
+{
+	struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
+	struct hwd_affine_reg *reg;
+	int i;
+
+	reg = res->reg;
+
+	/* Initialize status */
+	res->status.comp_syntax_error = 0;
+	res->status.comp_over_coordinate = 0;
+	res->status.comp_over_data = 0;
+	res->status.comp_lack_data = 0;
+	res->status.comp_size = 0;
+
+	/* Clear interrupt status */
+	writel(HWD_AFFINE_CLEAR_BITS, &reg->statc);
+
+	/* Reset T_RST bit, C_RST bit, and O_RST bits*/
+	writel(desc->ctrl, &reg->ctrl);
+	readl(&reg->ctrl);
+
+	writel(desc->src_size, &reg->src_size);
+	writel((u32)(desc->src_addr & HWD_AFFINE_HWD_MASK_ADDR), &reg->src_addr);
+	writel(desc->dst_bgn, &reg->dst_bgn);
+	writel(desc->dst_size, &reg->dst_size);
+	writel((u32)(desc->dst_addr & HWD_AFFINE_HWD_MASK_ADDR), &reg->dst_addr);
+	writel(desc->l_ptch, &reg->l_ptch);
+
+	for (i = 0; i < NUM_AFFINE_PARAMS; i++)
+		writel(desc->affine_param[i], &reg->affine_param[i]);
+
+	for (i = 0; i < NUM_HOMO_PARAMS; i++)
+		writel(desc->homo_param[i], &reg->homo_param[i]);
+
+	/* AFFINE Table */
+	writel((u32)(desc->tbl_addr & HWD_AFFINE_HWD_MASK_ADDR), &reg->tbl_addr);
+	writel(desc->tbl_ptch, &reg->tbl_ptch);
+	writel(desc->tbl_comp_size, &reg->tbl_comp_size);
+
+	for (i = 0; i < NUM_DISTORT_PARAMS; i++)
+		writel(desc->dist_param[i], &reg->dist_param[i]);
+
+	writel(HWD_AFFINE_INTMASK_NORMAL, &reg->intm);
+
+	writel(desc->mode, &reg->mode);
+	writel(desc->mode2, &reg->mode2);
+	writel(desc->adj_p, &reg->adj_p);
+
+	/* Data Sync */
+	dsb(st);
+
+	/* Kick AFFINE */
+	writel(HWD_AFFINE_START_CMD, &reg->ctrl);
+}
+
+/**
+ * hwd_affine_irq_handler() - HWD AFFINE interrupt handler
+ *
+ * @module_id: @ref hwd_affine_device "id" of the h/w module
+ * Return: following bitfields
+ * - HWD_AFFINE_EVENT_DONE: HW processing is completed
+ * - HWD_AFFINE_EVENT_SYNTAX_ERROR: HW Table Syntax error
+ * - HWD_AFFINE_EVENT_OVER_COORD: HW Table decoding coordinate error
+ * - HWD_AFFINE_EVENT_OVER_DATA: HW Table over compressed data error
+ * - HWD_AFFINE_EVENT_LACK_DATA: HW Table compressed data insufficient error
+ * - HWD_AFFINE_EVENT_SIZE_ERROR: HW Table smaller than the size actual size of compressed data
+ */
+u32 hwd_affine_irq_handler(u32 module_id)
+{
+	u32 event = 0;
+	struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
+	struct hwd_affine_reg *reg;
+	u32 cause;
+
+	reg = res->reg;
+
+	/* Read and Clear the interrupt causes */
+	cause = readl(&reg->statc);
+	writel(cause, &reg->statc);
+
+	if (cause & HWD_AFFINE_INT_END)
+		/* Execution End */
+		event |= HWD_AFFINE_EVENT_DONE;
+
+	if (cause & HWD_AFFINE_INT_TCOMP_ERR0) {
+		/* Syntax error */
+		res->status.comp_syntax_error = 1;
+		event |= HWD_AFFINE_EVENT_SYNTAX_ERROR;
+	}
+
+	if (cause & HWD_AFFINE_INT_TCOMP_ERR1) {
+		/* Decoding coordinate error */
+		res->status.comp_over_coordinate = 1;
+		event |= HWD_AFFINE_EVENT_OVER_COORD;
+	}
+
+	if (cause & HWD_AFFINE_INT_TCOMP_ERR2) {
+		/* Over compressed data error */
+		res->status.comp_over_data = 1;
+		event |= HWD_AFFINE_EVENT_OVER_DATA;
+	}
+
+	if (cause & HWD_AFFINE_INT_TCOMP_ERR3) {
+		/* Compressed data insufficient error */
+		res->status.comp_lack_data = 1;
+		event |= HWD_AFFINE_EVENT_LACK_DATA;
+	}
+
+	if (cause & HWD_AFFINE_INT_TCOMP_SIZE) {
+		/* Smaller than the actual size of compressed data */
+		res->status.comp_size = 1;
+		event |= HWD_AFFINE_EVENT_SIZE_ERROR;
+	}
+
+	return event;
+}
+
+/**
+ * hwd_affine_get_status() - HWD AFFINE Get Status
+ *
+ * @module_id: @ref hwd_affine_device "id" of the h/w module
+ * @status: Pointer to status structure
+ * Return: RETURN_OK operation completed successfully
+ */
+void hwd_affine_get_status(u32 module_id, struct hwd_affine_status *status)
+{
+	const struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
+
+	*status = res->status;
+}
diff --git a/drivers/soc/visconti/affine/hwd_affine.h b/drivers/soc/visconti/affine/hwd_affine.h
new file mode 100644
index 000000000..ed17b1116
--- /dev/null
+++ b/drivers/soc/visconti/affine/hwd_affine.h
@@ -0,0 +1,83 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/* Toshiba Visconti Affine Accelerator Support
+ *
+ * (C) Copyright 2022 TOSHIBA CORPORATION
+ * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#ifndef HWD_AFFINE_H
+#define HWD_AFFINE_H
+
+/**
+ * enum hwd_affine_device_id - AFFINE HWD Device ID
+ */
+enum hwd_affine_device_id {
+	HWD_AFFINE_DEVICE_0 = 0,
+	HWD_AFFINE_DEVICE_1 = 1,
+	HWD_AFFINE_DEVICE_MAX = 2,
+};
+
+/* hwd_affine_event_flags - Macro for Affine driver events */
+#define HWD_AFFINE_EVENT_DONE	      BIT(0)
+#define HWD_AFFINE_EVENT_SYNTAX_ERROR BIT(1)
+#define HWD_AFFINE_EVENT_OVER_COORD   BIT(2)
+#define HWD_AFFINE_EVENT_OVER_DATA    BIT(3)
+#define HWD_AFFINE_EVENT_LACK_DATA    BIT(4)
+#define HWD_AFFINE_EVENT_SIZE_ERROR   BIT(5)
+#define HWD_AFFINE_EVENT_MAX	      BIT(6)
+
+/* hwd_affine_param_num_flags Macro for Affine parameter number */
+#define HWD_AFFINE_AFFINE_PARAM_NUM	 (6U)
+#define HWD_AFFINE_UNDIST_PARAM_NUM	 (12U)
+#define HWD_AFFINE_HOMO_PARAM_NUM	 (3U)
+#define HWD_AFFINE_TEMP_UNDIST_PARAM_NUM (10U)
+
+/**
+ * struct hwd_affine_status - HWD Affine status
+ * @comp_size: Setting size is smaller than actual size
+ * @comp_lack_data: Compressed table is lacking enough data for processing
+ * @comp_over_data: Compressed table is exceeding the limit of data that can be processed
+ * @comp_over_coordinate: Number of coordinates in compressed table data exceeded the limit
+ * @comp_syntax_error: Syntax error of compressed table has occurred
+ * @reserved: Padding
+ */
+struct hwd_affine_status {
+	u32 comp_size : 1;
+	u32 comp_lack_data : 1;
+	u32 comp_over_data : 1;
+	u32 comp_over_coordinate : 1;
+	u32 comp_syntax_error : 1;
+	u32 reserved : 27;
+};
+
+/**
+ * struct hwd_affine_descriptor - HWD AFFINE Descriptor
+ */
+struct hwd_affine_descriptor {
+	u32 ctrl;
+	u32 mode2;
+	u32 mode;
+	u32 adj_p;
+	u32 src_size;
+	u32 src_addr;
+	u32 dst_bgn;
+	u32 dst_size;
+	u32 dst_addr;
+	u32 l_ptch;
+	u32 tbl_addr;
+	u32 tbl_ptch;
+	u32 affine_param[HWD_AFFINE_AFFINE_PARAM_NUM];
+	u32 dist_param[HWD_AFFINE_UNDIST_PARAM_NUM];
+	u32 homo_param[HWD_AFFINE_HOMO_PARAM_NUM];
+	u32 tbl_comp_size;
+	float temp_dist_p[HWD_AFFINE_TEMP_UNDIST_PARAM_NUM];
+	u16 config_done;
+};
+
+void hwd_affine_initialize(u32 module_id, void *vaddr);
+void hwd_affine_uninitialize(u32 module_id);
+void hwd_affine_start(u32 module_id, const struct hwd_affine_descriptor *desc);
+u32 hwd_affine_irq_handler(u32 module_id);
+void hwd_affine_get_status(u32 module_id, struct hwd_affine_status *status);
+
+#endif /* HWD_AFFINE_H */
diff --git a/drivers/soc/visconti/affine/hwd_affine_reg.h b/drivers/soc/visconti/affine/hwd_affine_reg.h
new file mode 100644
index 000000000..407d898fa
--- /dev/null
+++ b/drivers/soc/visconti/affine/hwd_affine_reg.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/* Toshiba Visconti Affine Accelerator Support
+ *
+ * (C) Copyright 2022 TOSHIBA CORPORATION
+ * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#ifndef HWD_AFFINE_REG_H
+#define HWD_AFFINE_REG_H
+
+/* hwd_affine_params_num_flags - Macro for Affine parameter number */
+#define HWD_AFFINE_AFFINE_PARAMS_NUM (6)
+#define HWD_AFFINE_UNDIST_PARAMS_NUM (12)
+#define HWD_AFFINE_HOMO_PARAMS_NUM   (3)
+#define HWD_AFFINE_RESERVED	     (472)
+
+/**
+ * struct hwd_affine_reg - Structure of Affine register descriptor
+ */
+struct hwd_affine_reg {
+	u32 ctrl;
+	u32 reserved0[2];
+	u32 mode2;
+	u32 mode;
+	u32 adj_p;
+	u32 intm;
+	u32 statc;
+	u32 out_lpos;
+	u32 src_size;
+	u32 src_addr;
+	u32 dst_bgn;
+	u32 dst_size;
+	u32 dst_addr;
+	u32 l_ptch;
+	u32 tbl_addr;
+	u32 tbl_ptch;
+	u32 affine_param[HWD_AFFINE_AFFINE_PARAMS_NUM];
+	u32 dist_param[HWD_AFFINE_UNDIST_PARAMS_NUM];
+	u32 reserved1;
+	u32 homo_param[HWD_AFFINE_HOMO_PARAMS_NUM];
+	u32 tbl_comp_size;
+	u32 reserved2[HWD_AFFINE_RESERVED];
+};
+
+#endif /* HWD_AFFINE_REG_H */
diff --git a/drivers/soc/visconti/uapi/affine.h b/drivers/soc/visconti/uapi/affine.h
new file mode 100644
index 000000000..539261483
--- /dev/null
+++ b/drivers/soc/visconti/uapi/affine.h
@@ -0,0 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * (C) Copyright 2020 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#ifndef _UAPI_LINUX_AFIINE_H
+#define _UAPI_LINUX_AFIINE_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+#include "ipa.h"
+
+#define DRV_AFFINE_BIT_CONFIG_DESC_FINAL (0x8000U)
+#define DRV_AFFINE_BUFFER_INDEX_MAX	 (3)
+
+#define DRV_AFFINE_AFFINE_PARAM_NUM	 (6U)
+#define DRV_AFFINE_UNDIST_PARAM_NUM	 (12U)
+#define DRV_AFFINE_HOMO_PARAM_NUM	 (3U)
+#define DRV_AFFINE_TEMP_UNDIST_PARAM_NUM (10U)
+
+/**
+ * struct drv_affine_status - AFFINE IPA status for IOC_IPA_GET_STATUS
+ * @state:                State of driver
+ * @comp_size:            Setting size is smaller than actual size
+ * @comp_lack_data:       Compressed table is lacking enough data for processing
+ * @comp_over_data:       Compressed table is exceeding the limit of data that can be processed
+ * @comp_over_coordinate: Number of coordinates in compressed table data exceeded the limit
+ * @comp_syntax_error:    Syntax error of compressed table has occurred
+ * @reserved:             Padding
+ */
+struct drv_affine_status {
+	enum drv_ipa_state state;
+	uint32_t comp_size : 1;
+	uint32_t comp_lack_data : 1;
+	uint32_t comp_over_data : 1;
+	uint32_t comp_over_coordinate : 1;
+	uint32_t comp_syntax_error : 1;
+	uint32_t reserved : 27;
+};
+
+/**
+ * struct drv_affine_descriptor - AFFINE IPA descriptor for IOC_IPA_START
+ * @ctrl:            Operation control of AFFINE
+ * @mode2:           Operation mode for the AFFINE execution
+ * @mode:            Operation mode for the AFFINE execution
+ * @adj_p:           Output Pixel Value Adjustment
+ * @src_size:        Input image size
+ * @src_addr:        Start address of input image
+ * @dst_bgn:         Start coordinate of the output rectangle area
+ * @dst_size:        Size of the output rectangle area
+ * @dst_addr:        Address of the output rectangle area
+ * @l_ptch:          Line pitch of both the input image and output rectangle area
+ * @tbl_addr:        Start address of the table used in the table conversion mode.
+ * @tbl_ptch:        Line pitch of the table used in the table conversion mode
+ * @affine_param:    Parameters for Affine transformation
+ * @dist_param:      Parameters for Distortion correction
+ * @homo_param:      Parameters Homography transformation
+ * @tbl_comp_size:   Table compression size
+ * @temp_dist_p:     Temporary buffer to hold distortion input parameters
+ * @config_done:     Flags of called configuration
+ * @buffer_info:     Table of buffer information
+ * @buffer_info_num: Number of buffer_info
+ */
+struct drv_affine_descriptor {
+	uint32_t ctrl;
+	uint32_t mode2;
+	uint32_t mode;
+	uint32_t adj_p;
+	uint32_t src_size;
+	struct drv_ipa_addr src_addr;
+	uint32_t dst_bgn;
+	uint32_t dst_size;
+	struct drv_ipa_addr dst_addr;
+	uint32_t l_ptch;
+	struct drv_ipa_addr tbl_addr;
+	uint32_t tbl_ptch;
+	uint32_t affine_param[DRV_AFFINE_AFFINE_PARAM_NUM];
+	uint32_t dist_param[DRV_AFFINE_UNDIST_PARAM_NUM];
+	uint32_t homo_param[DRV_AFFINE_HOMO_PARAM_NUM];
+	uint32_t tbl_comp_size;
+	float temp_dist_p[DRV_AFFINE_TEMP_UNDIST_PARAM_NUM];
+	uint16_t config_done;
+	struct drv_ipa_buffer_info buffer_info[DRV_AFFINE_BUFFER_INDEX_MAX];
+	int32_t buffer_info_num;
+};
+
+#endif /* _UAPI_LINUX_AFIINE_H */
-- 
2.17.1

Re: [PATCH v2 3/4] soc: visconti: Add Toshiba Visconti AFFINE image processing accelerator

[Hans Verkuil]

On 4/27/22 15:23, Yuji Ishikawa wrote:
> Adds support to AFFINE image processing accelerator on Toshiba Visconti ARM SoCs.
> This accelerator supoorts affine transform, lens undistortion and LUT transform.
> 
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> ---
> v1 -> v2:
>   - apply checkpatch.pl --strict
>   - renamed identifiers; hwd_AFFINE_xxxx to hwd_affine_xxxx
> ---
>  drivers/soc/visconti/Kconfig                 |   6 +
>  drivers/soc/visconti/Makefile                |   2 +
>  drivers/soc/visconti/affine/Makefile         |   6 +
>  drivers/soc/visconti/affine/affine.c         | 451 +++++++++++++++++++
>  drivers/soc/visconti/affine/hwd_affine.c     | 206 +++++++++
>  drivers/soc/visconti/affine/hwd_affine.h     |  83 ++++
>  drivers/soc/visconti/affine/hwd_affine_reg.h |  45 ++
>  drivers/soc/visconti/uapi/affine.h           |  87 ++++
>  8 files changed, 886 insertions(+)
>  create mode 100644 drivers/soc/visconti/affine/Makefile
>  create mode 100644 drivers/soc/visconti/affine/affine.c
>  create mode 100644 drivers/soc/visconti/affine/hwd_affine.c
>  create mode 100644 drivers/soc/visconti/affine/hwd_affine.h
>  create mode 100644 drivers/soc/visconti/affine/hwd_affine_reg.h
>  create mode 100644 drivers/soc/visconti/uapi/affine.h
> 
> diff --git a/drivers/soc/visconti/Kconfig b/drivers/soc/visconti/Kconfig
> index 8b1378917..01583d407 100644
> --- a/drivers/soc/visconti/Kconfig
> +++ b/drivers/soc/visconti/Kconfig
> @@ -1 +1,7 @@
> +if ARCH_VISCONTI
> +
> +config VISCONTI_AFFINE
> +    bool "Visconti Affine driver"
> +
> +endif
>  
> diff --git a/drivers/soc/visconti/Makefile b/drivers/soc/visconti/Makefile
> index 8d710da08..b25a726c3 100644
> --- a/drivers/soc/visconti/Makefile
> +++ b/drivers/soc/visconti/Makefile
> @@ -4,3 +4,5 @@
>  #
>  
>  obj-y += ipa_common.o
> +
> +obj-$(CONFIG_VISCONTI_AFFINE) += affine/
> diff --git a/drivers/soc/visconti/affine/Makefile b/drivers/soc/visconti/affine/Makefile
> new file mode 100644
> index 000000000..82f83b2d6
> --- /dev/null
> +++ b/drivers/soc/visconti/affine/Makefile
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for the Visconti AFFINE driver
> +#
> +
> +obj-y += affine.o hwd_affine.o
> diff --git a/drivers/soc/visconti/affine/affine.c b/drivers/soc/visconti/affine/affine.c
> new file mode 100644
> index 000000000..eea045dcf
> --- /dev/null
> +++ b/drivers/soc/visconti/affine/affine.c
> @@ -0,0 +1,451 @@
> +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
> +/* Toshiba Visconti Affine Accelerator Support
> + *
> + * (C) Copyright 2022 TOSHIBA CORPORATION
> + * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
> + */
> +
> +#include <linux/delay.h>
> +#include <linux/init.h>
> +#include <linux/interrupt.h>
> +#include <linux/iommu.h>
> +#include <linux/kernel.h>
> +#include <linux/miscdevice.h>
> +#include <linux/module.h>
> +#include <linux/mutex.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +#include <linux/poll.h>
> +#include <linux/wait.h>
> +
> +#include "hwd_affine.h"
> +#include "../ipa_common.h"
> +#include "../uapi/affine.h"
> +
> +struct affine_priv {
> +	struct device *dev;
> +	struct miscdevice miscdev;
> +	struct mutex lock;
> +	void __iomem *regs;
> +	int irq;
> +	wait_queue_head_t waitq;
> +	enum drv_ipa_state status;
> +	unsigned int hwd_event;
> +	unsigned int poll_event;
> +	int id;
> +	char name[16];
> +	bool dma_coherent;
> +	struct hwd_affine_status hwd_status;
> +
> +	struct dma_buf_attachment *dba[DRV_AFFINE_BUFFER_INDEX_MAX];
> +	struct sg_table *sgt[DRV_AFFINE_BUFFER_INDEX_MAX];
> +	enum dma_data_direction dma_dir[DRV_AFFINE_BUFFER_INDEX_MAX];
> +	unsigned int dma_count;
> +
> +	dma_addr_t buffer_iova[DRV_AFFINE_BUFFER_INDEX_MAX];
> +};
> +
> +static u32 affine_ipa_addr_to_iova(struct affine_priv *priv, struct drv_ipa_addr addr)
> +{
> +	u32 iova = 0;
> +
> +	if (addr.buffer_index < priv->dma_count &&
> +	    addr.offset < priv->dba[addr.buffer_index]->dmabuf->size)
> +		iova = priv->buffer_iova[addr.buffer_index] + addr.offset;
> +	return iova;
> +}
> +
> +static int affine_attach_dma_buf(struct affine_priv *priv, unsigned int buffer_index,
> +				 struct drv_ipa_buffer_info *buffer_info)
> +{
> +	int ret = 0;
> +	dma_addr_t addr;
> +
> +	if (buffer_index >= DRV_AFFINE_BUFFER_INDEX_MAX) {
> +		dev_err(priv->dev, "Buffer index invalid: index=%d\n", buffer_index);
> +		return -EINVAL;
> +	}
> +
> +	switch (buffer_info[buffer_index].direction) {
> +	case DRV_IPA_DIR_NONE:
> +		priv->dma_dir[priv->dma_count] = DMA_NONE;
> +		break;
> +	case DRV_IPA_DIR_TO_DEVICE:
> +		priv->dma_dir[priv->dma_count] = DMA_TO_DEVICE;
> +		break;
> +	case DRV_IPA_DIR_FROM_DEVICE:
> +		priv->dma_dir[priv->dma_count] = DMA_FROM_DEVICE;
> +		break;
> +	case DRV_IPA_DIR_BIDIRECTION:
> +		priv->dma_dir[priv->dma_count] = DMA_BIDIRECTIONAL;
> +		break;
> +	default:
> +		dev_err(priv->dev, "DMA direction invalid: index=%d dir=%d\n", buffer_index,
> +			buffer_info[buffer_index].direction);
> +		return -EINVAL;
> +	}
> +
> +	if (!buffer_info[buffer_index].coherent) {
> +		priv->dev->dma_coherent = false;
> +		if (priv->dma_coherent)
> +			priv->dma_dir[priv->dma_count] = DMA_NONE;
> +	}
> +
> +	ret = ipa_attach_dmabuf(priv->dev, buffer_info[buffer_index].fd,
> +				&priv->dba[priv->dma_count], &priv->sgt[priv->dma_count], &addr,
> +				priv->dma_dir[priv->dma_count]);
> +	if (ret == 0) {
> +		priv->dma_count++;
> +		priv->buffer_iova[buffer_index] = addr;
> +	}
> +
> +	priv->dev->dma_coherent = priv->dma_coherent;
> +
> +	return ret;
> +}
> +
> +static void affine_detach_dma_buf(struct affine_priv *priv)
> +{
> +	struct dma_buf *dmabuf;
> +	int i;
> +
> +	for (i = 0; i < priv->dma_count; i++) {
> +		dmabuf = priv->dba[i]->dmabuf;
> +		dma_buf_unmap_attachment(priv->dba[i], priv->sgt[i], priv->dma_dir[i]);
> +		dma_buf_detach(dmabuf, priv->dba[i]);
> +		dma_buf_put(dmabuf);
> +	}
> +}
> +
> +static irqreturn_t affine_irq(int irq, void *dev_id)
> +{
> +	struct affine_priv *priv = dev_id;
> +
> +	priv->hwd_event = hwd_affine_irq_handler(priv->id);
> +
> +	disable_irq_nosync(priv->irq);
> +
> +	return IRQ_WAKE_THREAD;
> +}
> +
> +static irqreturn_t affine_irq_thread(int irq, void *dev_id)
> +{
> +	struct affine_priv *priv = dev_id;
> +	unsigned long delay = 1;
> +
> +	mutex_lock(&priv->lock);
> +	affine_detach_dma_buf(priv);
> +
> +	hwd_affine_get_status(priv->id, &priv->hwd_status);
> +
> +	priv->status = DRV_IPA_STATE_IDLE;
> +
> +	/* status should be updated before poll_event so that
> +	 * when poll() returns, user context must observe state as idle
> +	 */
> +	smp_wmb();
> +
> +	if (priv->hwd_event == HWD_AFFINE_EVENT_DONE)
> +		priv->poll_event = IPA_POLL_EVENT_DONE;
> +	else
> +		priv->poll_event = IPA_POLL_EVENT_ERROR;
> +
> +	/* General barrier to avoid re-ordering of priv->poll_event=N and
> +	 * waitqueue_active()
> +	 */
> +	smp_mb();
> +
> +	/* Threads going to sleep in poll() can miss wakeup, when wakeup is done
> +	 * between event check in ipa_poll() and sleeping. Wakeup repeatedly.
> +	 */
> +	while (waitqueue_active(&priv->waitq)) {
> +		wake_up_interruptible(&priv->waitq);
> +
> +		WARN_ON(delay > IPA_WAKEUP_RETRY_DELAY);
> +		usleep_range(delay, delay + 1);
> +		delay += delay;
> +	}
> +
> +	mutex_unlock(&priv->lock);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static void affine_start(struct affine_priv *priv, struct hwd_affine_descriptor *desc)
> +{
> +	hwd_affine_start(priv->id, desc);
> +}
> +
> +static int affine_ioctl_start(struct affine_priv *priv, unsigned long arg)
> +{
> +	struct hwd_affine_descriptor hwd_desc;
> +	struct drv_affine_descriptor desc;
> +	int ret = 0;
> +	int i;
> +
> +	ret = mutex_lock_interruptible(&priv->lock);
> +	if (ret)
> +		return ret;
> +
> +	if (priv->status == DRV_IPA_STATE_BUSY) {
> +		dev_dbg(priv->dev, "busy: %d\n", priv->status);
> +		ret = -EBUSY;
> +		goto err1;
> +	}
> +
> +	if (copy_from_user(&desc, (void __user *)arg, sizeof(struct drv_affine_descriptor))) {
> +		dev_err(priv->dev, "Descriptor memory access error\n");
> +		ret = -EFAULT;
> +		goto err1;
> +	}
> +
> +	if (DRV_AFFINE_BIT_CONFIG_DESC_FINAL !=
> +	    (desc.config_done & DRV_AFFINE_BIT_CONFIG_DESC_FINAL)) {
> +		dev_err(priv->dev, "Descriptor configuration not complete\n");
> +		ret = -EINVAL;
> +		goto err1;
> +	}
> +
> +	priv->dma_count = 0;
> +
> +	/* setup buffer */
> +	for (i = 0; i < desc.buffer_info_num; i++) {
> +		ret = affine_attach_dma_buf(priv, i, desc.buffer_info);
> +		if (ret) {
> +			dev_err(priv->dev, "dma buf attach error: index=%d\n", i);
> +			goto err2;
> +		}
> +		dev_dbg(priv->dev, "@buffer[%d]@: fd=%d %s iova=%llx\n", i, desc.buffer_info[i].fd,
> +			desc.buffer_info[i].coherent ? "coherent" : "non-coherent",
> +			(uint64_t)priv->buffer_iova[i]);
> +	}
> +
> +	memcpy(&hwd_desc, &desc, sizeof(struct hwd_affine_descriptor));
> +
> +	hwd_desc.src_addr = affine_ipa_addr_to_iova(priv, desc.src_addr);
> +	if (hwd_desc.src_addr == 0) {
> +		dev_err(priv->dev, "IPA address to iova conversion error: src_addr %s: %d\n",
> +			__func__, __LINE__);
> +		ret = -EINVAL;
> +		goto err2;
> +	}
> +
> +	hwd_desc.dst_addr = affine_ipa_addr_to_iova(priv, desc.dst_addr);
> +	if (hwd_desc.dst_addr == 0) {
> +		dev_err(priv->dev, "IPA address to iova conversion error: dst_addr %s: %d\n",
> +			__func__, __LINE__);
> +		ret = -EINVAL;
> +		goto err2;
> +	}
> +
> +	hwd_desc.tbl_addr = 0;
> +	if (desc.tbl_ptch != 0) {
> +		hwd_desc.tbl_addr = affine_ipa_addr_to_iova(priv, desc.tbl_addr);
> +		if (hwd_desc.tbl_addr == 0) {
> +			dev_err(priv->dev,
> +				"IPA address to iova conversion error: tbl_addr %s: %d\n", __func__,
> +				__LINE__);
> +			ret = -EINVAL;
> +			goto err2;
> +		}
> +	}
> +
> +	dev_dbg(priv->dev, "src: 0x%x\n", hwd_desc.src_addr);
> +	dev_dbg(priv->dev, "dst: 0x%x\n", hwd_desc.dst_addr);
> +	dev_dbg(priv->dev, "tbl: 0x%x\n", hwd_desc.tbl_addr);
> +
> +	affine_start(priv, &hwd_desc);
> +
> +	priv->poll_event = IPA_POLL_EVENT_NONE;
> +	priv->hwd_event = 0;
> +	priv->status = DRV_IPA_STATE_BUSY;
> +	/* Barrier to prevent affine_irq() from setting priv->hwd_event
> +	 * before it is reset above
> +	 */
> +	smp_wmb();
> +	enable_irq(priv->irq);
> +
> +	mutex_unlock(&priv->lock);
> +
> +	return ret;
> +
> +err2:
> +	affine_detach_dma_buf(priv);
> +err1:
> +	mutex_unlock(&priv->lock);
> +	return ret;
> +}
> +
> +static int affine_ioctl_get_status(struct affine_priv *priv, unsigned long arg)
> +{
> +	struct drv_affine_status status;
> +	struct hwd_affine_status hwd_status;
> +	int ret = 0;
> +
> +	ret = mutex_lock_interruptible(&priv->lock);
> +	if (ret)
> +		return ret;
> +
> +	if (priv->status == DRV_IPA_STATE_BUSY)
> +		hwd_affine_get_status(priv->id, &hwd_status);
> +	else
> +		hwd_status = priv->hwd_status;
> +
> +	status.state = priv->status;
> +	mutex_unlock(&priv->lock);
> +
> +	status.comp_lack_data = hwd_status.comp_lack_data;
> +	status.comp_over_coordinate = hwd_status.comp_over_coordinate;
> +	status.comp_over_data = hwd_status.comp_over_data;
> +	status.comp_size = hwd_status.comp_size;
> +	status.comp_syntax_error = hwd_status.comp_syntax_error;
> +	if (copy_to_user((void __user *)arg, &status, sizeof(struct drv_affine_status))) {
> +		dev_err(priv->dev, "status memory access error\n");
> +		ret = -EFAULT;
> +	}
> +
> +	return ret;
> +}
> +
> +static long affine_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
> +{
> +	struct affine_priv *priv = container_of(fp->private_data, struct affine_priv, miscdev);
> +	int ret = 0;
> +
> +	switch (cmd) {
> +	case IOC_IPA_START:
> +		ret = affine_ioctl_start(priv, arg);
> +		break;
> +	case IOC_IPA_GET_STATUS:
> +		ret = affine_ioctl_get_status(priv, arg);
> +		break;
> +	default:
> +		ret = -ENOIOCTLCMD;
> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +static __poll_t affine_poll(struct file *fp, poll_table *wait)
> +{
> +	struct affine_priv *priv = container_of(fp->private_data, struct affine_priv, miscdev);
> +	__poll_t mask = 0;
> +	unsigned int poll_event;
> +
> +	poll_wait(fp, &priv->waitq, wait);
> +
> +	/* Barrier to avoid re-ordering of poll_wait() and event load
> +	 * Read barrier here and release barrier in poll_wait() together will
> +	 * prevent re-ordering
> +	 */
> +	smp_rmb();
> +	poll_event = priv->poll_event;
> +	if (poll_event != IPA_POLL_EVENT_NONE) {
> +		if (poll_event == IPA_POLL_EVENT_DONE)
> +			mask = EPOLLIN | EPOLLRDNORM;
> +		else
> +			mask = EPOLLERR;
> +	}
> +	return mask;
> +}
> +
> +static const struct file_operations affine_fops = {
> +	.owner = THIS_MODULE,
> +	.unlocked_ioctl = affine_ioctl,
> +	.poll = affine_poll,
> +};
> +
> +static int affine_probe(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct affine_priv *priv;
> +	int ret;
> +
> +	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
> +	if (!priv)
> +		return -ENOMEM;
> +
> +	mutex_init(&priv->lock);
> +
> +	/* update DMA mask */
> +	priv->dma_coherent = dev->dma_coherent;
> +	ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(36));
> +	if (ret)
> +		return ret;
> +
> +	priv->regs = devm_platform_ioremap_resource(pdev, 0);
> +	if (IS_ERR(priv->regs))
> +		return PTR_ERR(priv->regs);
> +
> +	priv->irq = platform_get_irq(pdev, 0);
> +	if (priv->irq < 0) {
> +		dev_err(dev, "failed to acquire irq resource\n");
> +		return -ENOENT;
> +	}
> +	ret = devm_request_threaded_irq(dev, priv->irq, affine_irq, affine_irq_thread, 0, "affine",
> +					priv);
> +	if (ret) {
> +		dev_err(dev, "irq request failed\n");
> +		return ret;
> +	}
> +	disable_irq(priv->irq);
> +
> +	ret = of_property_read_u32(dev->of_node, "index", &priv->id);
> +	if (ret) {
> +		dev_err(dev, "failed to acquire irq resource\n");
> +		return ret;
> +	}
> +
> +	hwd_affine_initialize(priv->id, priv->regs);
> +
> +	snprintf(priv->name, sizeof(priv->name), "affine%d", priv->id);
> +	priv->miscdev.minor = MISC_DYNAMIC_MINOR;
> +	priv->miscdev.name = priv->name;
> +	priv->miscdev.fops = &affine_fops;
> +	ret = misc_register(&priv->miscdev);
> +	if (ret) {
> +		dev_err(dev, "misc registration failed\n");
> +		hwd_affine_uninitialize(priv->id);
> +		return ret;
> +	}
> +
> +	priv->dev = dev;
> +	platform_set_drvdata(pdev, priv);
> +
> +	init_waitqueue_head(&priv->waitq);
> +
> +	priv->status = DRV_IPA_STATE_IDLE;
> +	return 0;
> +}
> +
> +static int affine_remove(struct platform_device *pdev)
> +{
> +	struct affine_priv *priv = platform_get_drvdata(pdev);
> +
> +	misc_deregister(&priv->miscdev);
> +	hwd_affine_uninitialize(priv->id);
> +	return 0;
> +}
> +
> +static const struct of_device_id affine_of_match[] = {
> +	{
> +		.compatible = "toshiba,visconti-affine",
> +	},
> +	{},
> +};
> +MODULE_DEVICE_TABLE(platform, affine_of_match);
> +
> +static struct platform_driver affine_driver = {
> +	.probe = affine_probe,
> +	.remove = affine_remove,
> +	.driver = {
> +		.name = "visconti_affine",
> +		.of_match_table = of_match_ptr(affine_of_match),
> +	},
> +};
> +module_platform_driver(affine_driver);
> +
> +MODULE_AUTHOR("Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>");
> +MODULE_DESCRIPTION("Toshiba Visconti affine driver");
> +MODULE_LICENSE("Dual BSD/GPL");
> diff --git a/drivers/soc/visconti/affine/hwd_affine.c b/drivers/soc/visconti/affine/hwd_affine.c
> new file mode 100644
> index 000000000..6720a0212
> --- /dev/null
> +++ b/drivers/soc/visconti/affine/hwd_affine.c
> @@ -0,0 +1,206 @@
> +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
> +/* Toshiba Visconti Affine Accelerator Support
> + *
> + * (C) Copyright 2022 TOSHIBA CORPORATION
> + * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
> + */
> +
> +#include <linux/delay.h>
> +#include <linux/io.h>
> +
> +#include "hwd_affine.h"
> +#include "hwd_affine_reg.h"
> +
> +#define HWD_AFFINE_CLEAR_BITS	  GENMASK(15, 0)
> +#define HWD_AFFINE_INTMASK_NORMAL (0x04100)
> +
> +#define HWD_AFFINE_INT_END	  BIT(0)
> +#define HWD_AFFINE_INT_TCOMP_ERR0 BIT(16)
> +#define HWD_AFFINE_INT_TCOMP_ERR1 BIT(17)
> +#define HWD_AFFINE_INT_TCOMP_ERR2 BIT(18)
> +#define HWD_AFFINE_INT_TCOMP_ERR3 BIT(19)
> +#define HWD_AFFINE_INT_TCOMP_SIZE BIT(20)
> +
> +#define HWD_AFFINE_HWD_MASK_ADDR GENMASK(31, 0)
> +
> +#define HWD_AFFINE_START_CMD BIT(0)
> +
> +/**
> + * struct hwd_affine_resources - HWD driver internal resource structure
> + */
> +struct hwd_affine_resources {
> +	struct hwd_affine_reg *reg;
> +	struct hwd_affine_status status;
> +};
> +
> +/* HWD driver internal resource */
> +static struct hwd_affine_resources hwd_affine_resources[HWD_AFFINE_DEVICE_MAX] = {};
> +
> +/**
> + * hwd_affine_initialize() - Initialize AFFINE device
> + *
> + * @module_id: @ref hwd_affine_device "id" of the h/w module
> + * @vaddr: register base virtual address
> + */
> +void hwd_affine_initialize(u32 module_id, void *vaddr)
> +{
> +	struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
> +
> +	/* Initialize the device */
> +	res->reg = (struct hwd_affine_reg *)vaddr;
> +}
> +
> +/**
> + * hwd_affine_uninitialize() - Uninitialize AFFINE device
> + *
> + * @module_id: @ref hwd_affine_device "id" of the h/w module
> + */
> +void hwd_affine_uninitialize(u32 module_id)
> +{
> +	struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
> +
> +	/* Deinitialize the device */
> +	res->reg = NULL;
> +}
> +
> +#define NUM_AFFINE_PARAMS  6
> +#define NUM_HOMO_PARAMS	   3
> +#define NUM_DISTORT_PARAMS 12
> +
> +/**
> + * hwd_affine_start() - Start AFFINE device
> + *
> + * @module_id: @ref hwd_affine_device "id" of the h/w module
> + * @desc: Pointer to AFFINE descriptor structure
> + */
> +void hwd_affine_start(u32 module_id, const struct hwd_affine_descriptor *desc)
> +{
> +	struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
> +	struct hwd_affine_reg *reg;
> +	int i;
> +
> +	reg = res->reg;
> +
> +	/* Initialize status */
> +	res->status.comp_syntax_error = 0;
> +	res->status.comp_over_coordinate = 0;
> +	res->status.comp_over_data = 0;
> +	res->status.comp_lack_data = 0;
> +	res->status.comp_size = 0;
> +
> +	/* Clear interrupt status */
> +	writel(HWD_AFFINE_CLEAR_BITS, &reg->statc);
> +
> +	/* Reset T_RST bit, C_RST bit, and O_RST bits*/
> +	writel(desc->ctrl, &reg->ctrl);
> +	readl(&reg->ctrl);
> +
> +	writel(desc->src_size, &reg->src_size);
> +	writel((u32)(desc->src_addr & HWD_AFFINE_HWD_MASK_ADDR), &reg->src_addr);
> +	writel(desc->dst_bgn, &reg->dst_bgn);
> +	writel(desc->dst_size, &reg->dst_size);
> +	writel((u32)(desc->dst_addr & HWD_AFFINE_HWD_MASK_ADDR), &reg->dst_addr);
> +	writel(desc->l_ptch, &reg->l_ptch);
> +
> +	for (i = 0; i < NUM_AFFINE_PARAMS; i++)
> +		writel(desc->affine_param[i], &reg->affine_param[i]);
> +
> +	for (i = 0; i < NUM_HOMO_PARAMS; i++)
> +		writel(desc->homo_param[i], &reg->homo_param[i]);
> +
> +	/* AFFINE Table */
> +	writel((u32)(desc->tbl_addr & HWD_AFFINE_HWD_MASK_ADDR), &reg->tbl_addr);
> +	writel(desc->tbl_ptch, &reg->tbl_ptch);
> +	writel(desc->tbl_comp_size, &reg->tbl_comp_size);
> +
> +	for (i = 0; i < NUM_DISTORT_PARAMS; i++)
> +		writel(desc->dist_param[i], &reg->dist_param[i]);
> +
> +	writel(HWD_AFFINE_INTMASK_NORMAL, &reg->intm);
> +
> +	writel(desc->mode, &reg->mode);
> +	writel(desc->mode2, &reg->mode2);
> +	writel(desc->adj_p, &reg->adj_p);
> +
> +	/* Data Sync */
> +	dsb(st);
> +
> +	/* Kick AFFINE */
> +	writel(HWD_AFFINE_START_CMD, &reg->ctrl);
> +}
> +
> +/**
> + * hwd_affine_irq_handler() - HWD AFFINE interrupt handler
> + *
> + * @module_id: @ref hwd_affine_device "id" of the h/w module
> + * Return: following bitfields
> + * - HWD_AFFINE_EVENT_DONE: HW processing is completed
> + * - HWD_AFFINE_EVENT_SYNTAX_ERROR: HW Table Syntax error
> + * - HWD_AFFINE_EVENT_OVER_COORD: HW Table decoding coordinate error
> + * - HWD_AFFINE_EVENT_OVER_DATA: HW Table over compressed data error
> + * - HWD_AFFINE_EVENT_LACK_DATA: HW Table compressed data insufficient error
> + * - HWD_AFFINE_EVENT_SIZE_ERROR: HW Table smaller than the size actual size of compressed data
> + */
> +u32 hwd_affine_irq_handler(u32 module_id)
> +{
> +	u32 event = 0;
> +	struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
> +	struct hwd_affine_reg *reg;
> +	u32 cause;
> +
> +	reg = res->reg;
> +
> +	/* Read and Clear the interrupt causes */
> +	cause = readl(&reg->statc);
> +	writel(cause, &reg->statc);
> +
> +	if (cause & HWD_AFFINE_INT_END)
> +		/* Execution End */
> +		event |= HWD_AFFINE_EVENT_DONE;
> +
> +	if (cause & HWD_AFFINE_INT_TCOMP_ERR0) {
> +		/* Syntax error */
> +		res->status.comp_syntax_error = 1;
> +		event |= HWD_AFFINE_EVENT_SYNTAX_ERROR;
> +	}
> +
> +	if (cause & HWD_AFFINE_INT_TCOMP_ERR1) {
> +		/* Decoding coordinate error */
> +		res->status.comp_over_coordinate = 1;
> +		event |= HWD_AFFINE_EVENT_OVER_COORD;
> +	}
> +
> +	if (cause & HWD_AFFINE_INT_TCOMP_ERR2) {
> +		/* Over compressed data error */
> +		res->status.comp_over_data = 1;
> +		event |= HWD_AFFINE_EVENT_OVER_DATA;
> +	}
> +
> +	if (cause & HWD_AFFINE_INT_TCOMP_ERR3) {
> +		/* Compressed data insufficient error */
> +		res->status.comp_lack_data = 1;
> +		event |= HWD_AFFINE_EVENT_LACK_DATA;
> +	}
> +
> +	if (cause & HWD_AFFINE_INT_TCOMP_SIZE) {
> +		/* Smaller than the actual size of compressed data */
> +		res->status.comp_size = 1;
> +		event |= HWD_AFFINE_EVENT_SIZE_ERROR;
> +	}
> +
> +	return event;
> +}
> +
> +/**
> + * hwd_affine_get_status() - HWD AFFINE Get Status
> + *
> + * @module_id: @ref hwd_affine_device "id" of the h/w module
> + * @status: Pointer to status structure
> + * Return: RETURN_OK operation completed successfully
> + */
> +void hwd_affine_get_status(u32 module_id, struct hwd_affine_status *status)
> +{
> +	const struct hwd_affine_resources *res = &hwd_affine_resources[module_id];
> +
> +	*status = res->status;
> +}
> diff --git a/drivers/soc/visconti/affine/hwd_affine.h b/drivers/soc/visconti/affine/hwd_affine.h
> new file mode 100644
> index 000000000..ed17b1116
> --- /dev/null
> +++ b/drivers/soc/visconti/affine/hwd_affine.h
> @@ -0,0 +1,83 @@
> +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
> +/* Toshiba Visconti Affine Accelerator Support
> + *
> + * (C) Copyright 2022 TOSHIBA CORPORATION
> + * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
> + */
> +
> +#ifndef HWD_AFFINE_H
> +#define HWD_AFFINE_H
> +
> +/**
> + * enum hwd_affine_device_id - AFFINE HWD Device ID
> + */
> +enum hwd_affine_device_id {
> +	HWD_AFFINE_DEVICE_0 = 0,
> +	HWD_AFFINE_DEVICE_1 = 1,
> +	HWD_AFFINE_DEVICE_MAX = 2,
> +};
> +
> +/* hwd_affine_event_flags - Macro for Affine driver events */
> +#define HWD_AFFINE_EVENT_DONE	      BIT(0)
> +#define HWD_AFFINE_EVENT_SYNTAX_ERROR BIT(1)
> +#define HWD_AFFINE_EVENT_OVER_COORD   BIT(2)
> +#define HWD_AFFINE_EVENT_OVER_DATA    BIT(3)
> +#define HWD_AFFINE_EVENT_LACK_DATA    BIT(4)
> +#define HWD_AFFINE_EVENT_SIZE_ERROR   BIT(5)
> +#define HWD_AFFINE_EVENT_MAX	      BIT(6)
> +
> +/* hwd_affine_param_num_flags Macro for Affine parameter number */
> +#define HWD_AFFINE_AFFINE_PARAM_NUM	 (6U)
> +#define HWD_AFFINE_UNDIST_PARAM_NUM	 (12U)
> +#define HWD_AFFINE_HOMO_PARAM_NUM	 (3U)
> +#define HWD_AFFINE_TEMP_UNDIST_PARAM_NUM (10U)
> +
> +/**
> + * struct hwd_affine_status - HWD Affine status
> + * @comp_size: Setting size is smaller than actual size
> + * @comp_lack_data: Compressed table is lacking enough data for processing
> + * @comp_over_data: Compressed table is exceeding the limit of data that can be processed
> + * @comp_over_coordinate: Number of coordinates in compressed table data exceeded the limit
> + * @comp_syntax_error: Syntax error of compressed table has occurred
> + * @reserved: Padding
> + */
> +struct hwd_affine_status {
> +	u32 comp_size : 1;
> +	u32 comp_lack_data : 1;
> +	u32 comp_over_data : 1;
> +	u32 comp_over_coordinate : 1;
> +	u32 comp_syntax_error : 1;
> +	u32 reserved : 27;
> +};
> +
> +/**
> + * struct hwd_affine_descriptor - HWD AFFINE Descriptor
> + */
> +struct hwd_affine_descriptor {
> +	u32 ctrl;
> +	u32 mode2;
> +	u32 mode;
> +	u32 adj_p;
> +	u32 src_size;
> +	u32 src_addr;
> +	u32 dst_bgn;
> +	u32 dst_size;
> +	u32 dst_addr;
> +	u32 l_ptch;
> +	u32 tbl_addr;
> +	u32 tbl_ptch;
> +	u32 affine_param[HWD_AFFINE_AFFINE_PARAM_NUM];
> +	u32 dist_param[HWD_AFFINE_UNDIST_PARAM_NUM];
> +	u32 homo_param[HWD_AFFINE_HOMO_PARAM_NUM];
> +	u32 tbl_comp_size;
> +	float temp_dist_p[HWD_AFFINE_TEMP_UNDIST_PARAM_NUM];
> +	u16 config_done;
> +};
> +
> +void hwd_affine_initialize(u32 module_id, void *vaddr);
> +void hwd_affine_uninitialize(u32 module_id);
> +void hwd_affine_start(u32 module_id, const struct hwd_affine_descriptor *desc);
> +u32 hwd_affine_irq_handler(u32 module_id);
> +void hwd_affine_get_status(u32 module_id, struct hwd_affine_status *status);
> +
> +#endif /* HWD_AFFINE_H */
> diff --git a/drivers/soc/visconti/affine/hwd_affine_reg.h b/drivers/soc/visconti/affine/hwd_affine_reg.h
> new file mode 100644
> index 000000000..407d898fa
> --- /dev/null
> +++ b/drivers/soc/visconti/affine/hwd_affine_reg.h
> @@ -0,0 +1,45 @@
> +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
> +/* Toshiba Visconti Affine Accelerator Support
> + *
> + * (C) Copyright 2022 TOSHIBA CORPORATION
> + * (C) Copyright 2022 Toshiba Electronic Devices & Storage Corporation
> + */
> +
> +#ifndef HWD_AFFINE_REG_H
> +#define HWD_AFFINE_REG_H
> +
> +/* hwd_affine_params_num_flags - Macro for Affine parameter number */
> +#define HWD_AFFINE_AFFINE_PARAMS_NUM (6)
> +#define HWD_AFFINE_UNDIST_PARAMS_NUM (12)
> +#define HWD_AFFINE_HOMO_PARAMS_NUM   (3)
> +#define HWD_AFFINE_RESERVED	     (472)
> +
> +/**
> + * struct hwd_affine_reg - Structure of Affine register descriptor
> + */
> +struct hwd_affine_reg {
> +	u32 ctrl;
> +	u32 reserved0[2];
> +	u32 mode2;
> +	u32 mode;
> +	u32 adj_p;
> +	u32 intm;
> +	u32 statc;
> +	u32 out_lpos;
> +	u32 src_size;
> +	u32 src_addr;
> +	u32 dst_bgn;
> +	u32 dst_size;
> +	u32 dst_addr;
> +	u32 l_ptch;
> +	u32 tbl_addr;
> +	u32 tbl_ptch;
> +	u32 affine_param[HWD_AFFINE_AFFINE_PARAMS_NUM];
> +	u32 dist_param[HWD_AFFINE_UNDIST_PARAMS_NUM];
> +	u32 reserved1;
> +	u32 homo_param[HWD_AFFINE_HOMO_PARAMS_NUM];
> +	u32 tbl_comp_size;
> +	u32 reserved2[HWD_AFFINE_RESERVED];
> +};
> +
> +#endif /* HWD_AFFINE_REG_H */
> diff --git a/drivers/soc/visconti/uapi/affine.h b/drivers/soc/visconti/uapi/affine.h
> new file mode 100644
> index 000000000..539261483
> --- /dev/null
> +++ b/drivers/soc/visconti/uapi/affine.h
> @@ -0,0 +1,87 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * (C) Copyright 2020 Toshiba Electronic Devices & Storage Corporation
> + */
> +
> +#ifndef _UAPI_LINUX_AFIINE_H

Typo: AFIINE -> AFFINE.

Same comment as for the DNN driver: there is no documentation whatsoever
about what it does or what data it expects.

Some high-level documentation is sorely needed.

Regards,

	Hans

> +#define _UAPI_LINUX_AFIINE_H
> +
> +#include <linux/ioctl.h>
> +#include <linux/types.h>
> +#include "ipa.h"
> +
> +#define DRV_AFFINE_BIT_CONFIG_DESC_FINAL (0x8000U)
> +#define DRV_AFFINE_BUFFER_INDEX_MAX	 (3)
> +
> +#define DRV_AFFINE_AFFINE_PARAM_NUM	 (6U)
> +#define DRV_AFFINE_UNDIST_PARAM_NUM	 (12U)
> +#define DRV_AFFINE_HOMO_PARAM_NUM	 (3U)
> +#define DRV_AFFINE_TEMP_UNDIST_PARAM_NUM (10U)
> +
> +/**
> + * struct drv_affine_status - AFFINE IPA status for IOC_IPA_GET_STATUS
> + * @state:                State of driver
> + * @comp_size:            Setting size is smaller than actual size
> + * @comp_lack_data:       Compressed table is lacking enough data for processing
> + * @comp_over_data:       Compressed table is exceeding the limit of data that can be processed
> + * @comp_over_coordinate: Number of coordinates in compressed table data exceeded the limit
> + * @comp_syntax_error:    Syntax error of compressed table has occurred
> + * @reserved:             Padding
> + */
> +struct drv_affine_status {
> +	enum drv_ipa_state state;
> +	uint32_t comp_size : 1;
> +	uint32_t comp_lack_data : 1;
> +	uint32_t comp_over_data : 1;
> +	uint32_t comp_over_coordinate : 1;
> +	uint32_t comp_syntax_error : 1;
> +	uint32_t reserved : 27;
> +};
> +
> +/**
> + * struct drv_affine_descriptor - AFFINE IPA descriptor for IOC_IPA_START
> + * @ctrl:            Operation control of AFFINE
> + * @mode2:           Operation mode for the AFFINE execution
> + * @mode:            Operation mode for the AFFINE execution
> + * @adj_p:           Output Pixel Value Adjustment
> + * @src_size:        Input image size
> + * @src_addr:        Start address of input image
> + * @dst_bgn:         Start coordinate of the output rectangle area
> + * @dst_size:        Size of the output rectangle area
> + * @dst_addr:        Address of the output rectangle area
> + * @l_ptch:          Line pitch of both the input image and output rectangle area
> + * @tbl_addr:        Start address of the table used in the table conversion mode.
> + * @tbl_ptch:        Line pitch of the table used in the table conversion mode
> + * @affine_param:    Parameters for Affine transformation
> + * @dist_param:      Parameters for Distortion correction
> + * @homo_param:      Parameters Homography transformation
> + * @tbl_comp_size:   Table compression size
> + * @temp_dist_p:     Temporary buffer to hold distortion input parameters
> + * @config_done:     Flags of called configuration
> + * @buffer_info:     Table of buffer information
> + * @buffer_info_num: Number of buffer_info
> + */
> +struct drv_affine_descriptor {
> +	uint32_t ctrl;
> +	uint32_t mode2;
> +	uint32_t mode;
> +	uint32_t adj_p;
> +	uint32_t src_size;
> +	struct drv_ipa_addr src_addr;
> +	uint32_t dst_bgn;
> +	uint32_t dst_size;
> +	struct drv_ipa_addr dst_addr;
> +	uint32_t l_ptch;
> +	struct drv_ipa_addr tbl_addr;
> +	uint32_t tbl_ptch;
> +	uint32_t affine_param[DRV_AFFINE_AFFINE_PARAM_NUM];
> +	uint32_t dist_param[DRV_AFFINE_UNDIST_PARAM_NUM];
> +	uint32_t homo_param[DRV_AFFINE_HOMO_PARAM_NUM];
> +	uint32_t tbl_comp_size;
> +	float temp_dist_p[DRV_AFFINE_TEMP_UNDIST_PARAM_NUM];
> +	uint16_t config_done;
> +	struct drv_ipa_buffer_info buffer_info[DRV_AFFINE_BUFFER_INDEX_MAX];
> +	int32_t buffer_info_num;
> +};
> +
> +#endif /* _UAPI_LINUX_AFIINE_H */

RE: [PATCH v2 3/4] soc: visconti: Add Toshiba Visconti AFFINE image processing accelerator

[yuji2.ishikawa]

Hi, Hans,

With your advice for DNN accelerator, I prepared some description of AFFINE IPA and its usage.

#### Handling memory blocks for Visconti5 accelerators (same description for DNN accelerator)

Visconti5 Image-Processing-Accelerators do not have fine grained IOMMU, as CPU have.
Therefore, memory region to be passed to the accelerators should be physically contiguous.
We use DMA-BUF backed by CMA (Contiguous Memory Allocator) to allocate memory regions for sharing between CPU/IPAs.
Originally, in v4.19 based implementation, the ION allocator was used to allocate DMA-BUF instances.
For the latest implementation, DMA-BUF HEAPS is used.

Two structure types are used to represent memory region passed to drivers.
* struct drv_ipa_buffer_info
  * to describe whole DMA-BUF instance
* struct drv_ipa_addr
  * to describe a memory region in a DMA-BUF instance

for details, see usage sample of each IPA driver


#### Image Processing Accelerators overview (same description for DNN accelerator)

Visconti5 SoC has following image processing accererators

* AFFINE: 1 input image, 1 output image;                                             Affine transform, Homography transform, Polynomial lens distortion, LUT transform
* DNN:    N input feature vector, N output feature vector;                           Deep neural network operation
* PYRAMID 3 input image, 3 * N output image;                                         Resize grayscale/color image with N different parameters
* DSPIF:  M input image, N output image;                                             Various opeations on images
* HOX:    1 input image (multi ROI), 1 input dictionary1 likelihood/feature vector;  Extended Histogram of Oriented Gradient based pattern matching
* HAMAT:  2 input feature vectors: 1 output corrdinate vector;                       Hamming distance matching for stereo vision
* FLMAT:  3 input image, N input feature point, N output matched point;              Optical flow matching
* SMLDB:  1 input image, N input feature point, N output feature vector;             Accelerated-KAZE feature descriptor accelerator
* STMAT:  2 input image, 1 output disparity image;                                   Stereo disparity

see [0] Fig 7.2.1 for block diagram (of prototype chip)


#### Affine Overview

AFFINE IPA is a proprietary image processing hardware developed by Toshiba.
Visconti5 SoC has 2 instances of AFFINE IPA hardware.
It has 4 operation modes:
  * Affine (Linear) transformation
  * Homography transformation
  * Polynomial lens distortion
  * LUT based transformation
It accepts 1 input image and yields 1 output image at an operation.


#### Input / Output

Input: 8bit grayscale or 16bit grayscale image
Output: 8bit grayscale or 16bit grayscale image

AFFINE IPA driver accepts an instance of "struct drv_affine_descriptor" which includes input/output images and operation parameters.


#### Descriptor Builder at userland

Following APIs are provided to build a descriptor instance at userland.

/* defined in drv_affine_util.h */
int32_t drv_AFFINE_config_descript_init(struct drv_affine_descriptor *desc, struct drv_ipa_buffer_info *buffer, int32_t buffer_num);
int32_t drv_AFFINE_config_input_image(struct drv_affine_descriptor *desc, struct drv_ipa_addr src_addr, int32_t src_width, int32_t src_height, int32_t src_pitch, int32_t src_depth);                                      
int32_t drv_AFFINE_config_output_image(struct drv_affine_descriptor *desc, struct drv_ipa_addr dst_addr, int32_t dst_width, int32_t dst_height, int32_t dst_offset_x, int32_t dst_offset_y, uint16_t fill_value, int32_t dst_pitch, int32_t dst_depth)
int32_t drv_AFFINE_config_interpolation_mode(...)
int32_t drv_AFFINE_config_linear(...)
int32_t drv_AFFINE_config_homography(...)
int32_t drv_AFFINE_config_undist(...)
int32_t drv_AFFINE_config_table(...)
int32_t drv_AFFINE_config_descript_finalize(struct drv_affine_descriptor *desc)

struct drv_affine_descriptor is defined in drivers/soc/visconti/uapi/affine.h.
I think this header should be placed anywhere else to be collected on "make headers_install" action of kernel building.


#### Usage sample (without error handlers)

    #include <linux/dma-heap.h>
    #include "drv_ipa.h" 
    #include "drv_affine.h"
    #include "drv_affine_util.h" 

    int allocate_buffer(int fd_heap, int size) 
    {
        struct dma_heap_allocation_data heap_data_in={0};
        int ret;

        heap_data_in.len = ROUNDUP_POW2(size);
        heap_data_in.fd_flags = O_RDWR | O_CLOEXEC;

        ret = ioctl(fd_heap, DMA_HEAP_IOCTL_ALLOC, &heap_data_in);
        if (ret <0)
            return -1;
        else
            return heap_data_in.fd;
    }

    void affine_sample(int fd_affine, int fd_src, int fd_dst) 
    {
        struct drv_ipa_buffer_info bufinfo[2] = {
            {.fd=fd_src, .coherent=true, .direction=DRV_IPA_DIR_TO_DEVICE},
            {.fd=fd_dst, .coherent=true, .direction=DRV_IPA_DIR_FROM_DEVICE},
        };
        struct drv_ipa_addr src_addr = {.buffer_index=0, .offset=0};
        struct drv_ipa_addr dst_addr = {.buffer_index=1, .offset=0};
        struct drv_affine_descriptor desc;

        drv_AFFINE_config_descript_init(&desc, bufinfo, 2);
        drv_AFFINE_config_input_image(&desc, src_addr, SRC_WIDTH, SRC_HEIGHT, SRC_PITCH, SRC_DEPTH_8);
        drv_AFFINE_config_output_image(&desc, dst_addr, DST_WIDTH, DST_HEIGHT, 0, 0, 0, DST_PITCH, DST_DEPTH_8);
        drv_AFFINE_config_interpolation_mode(&desc, DRV_AFFINE_BILINEAR, DRV_AFFINE_BICB_A_1_00);
        drv_AFFINE_config_linear(&desc, (float)(1.5), 0.0, 0.0, 0.0, (float)(1.5), 0.0, DRV_AFFINE_FIRST_ORDER);
        drv_AFFINE_config_descript_finalize(&desc);

        ioctl(fd_affine, IOC_IPA_START, &desc);

        {
            struct pollfd fds[] = {.fd=fd_affine, .events=POLL_IN, .revents = 0};
            poll(fds, 1, 1000);
        }
    }

    void sample()
    {
        int fd_affine, fd_heap, fd_src, fd_dst;
        fd_affine = open("/dev/affine0", O_RDWR);
        fd_heap   = open("/dev/dma_heap/linux,cma", O_RDWR);
        fd_src    = allocate_buffer(fd_heap, SRC_PITCH*SRC_HEIGHT);
        fd_dst    = allocate_buffer(fd_heap, DST_PITCH*DST_HEIGHT);

        /* fill initial value to src buffer here */

        affine_sample(fd_affine, fd_src, fd_dst);

        ...
    }


#### Reference

* [0] https://toshiba.semicon-storage.com/content/dam/toshiba-ss-v2/master/en/company/technical-review/pdf/technical-review-18_e.pdf
  * Fig 7.2.1 shows the whole architecture of prototype chip

Regards,
Yuji

> -----Original Message-----
> From: Hans Verkuil <hverkuil@xs4all.nl>
> Sent: Thursday, May 12, 2022 8:19 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>; Rob Herring <robh+dt@kernel.org>;
> iwamatsu nobuhiro(岩松 信洋 □ＳＷＣ◯ＡＣＴ)
> <nobuhiro1.iwamatsu@toshiba.co.jp>; Sumit Semwal
> <sumit.semwal@linaro.org>; Christian König <christian.koenig@amd.com>
> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> linux-media@vger.kernel.org; dri-devel@lists.freedesktop.org;
> linaro-mm-sig@lists.linaro.org
> Subject: Re: [PATCH v2 3/4] soc: visconti: Add Toshiba Visconti AFFINE image
> processing accelerator
> 
> On 4/27/22 15:23, Yuji Ishikawa wrote:
> > Adds support to AFFINE image processing accelerator on Toshiba Visconti
> ARM SoCs.
> > This accelerator supoorts affine transform, lens undistortion and LUT
> transform.
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> > ---
> > v1 -> v2:
> >   - apply checkpatch.pl --strict
> >   - renamed identifiers; hwd_AFFINE_xxxx to hwd_affine_xxxx
> > ---
> >  drivers/soc/visconti/Kconfig                 |   6 +
> >  drivers/soc/visconti/Makefile                |   2 +
> >  drivers/soc/visconti/affine/Makefile         |   6 +
> >  drivers/soc/visconti/affine/affine.c         | 451
> +++++++++++++++++++
> >  drivers/soc/visconti/affine/hwd_affine.c     | 206 +++++++++
> >  drivers/soc/visconti/affine/hwd_affine.h     |  83 ++++
> >  drivers/soc/visconti/affine/hwd_affine_reg.h |  45 ++
> >  drivers/soc/visconti/uapi/affine.h           |  87 ++++
> >  8 files changed, 886 insertions(+)
> >  create mode 100644 drivers/soc/visconti/affine/Makefile
> >  create mode 100644 drivers/soc/visconti/affine/affine.c
> >  create mode 100644 drivers/soc/visconti/affine/hwd_affine.c
> >  create mode 100644 drivers/soc/visconti/affine/hwd_affine.h
> >  create mode 100644 drivers/soc/visconti/affine/hwd_affine_reg.h
> >  create mode 100644 drivers/soc/visconti/uapi/affine.h
> >
> > diff --git a/drivers/soc/visconti/Kconfig
> > b/drivers/soc/visconti/Kconfig index 8b1378917..01583d407 100644
> > --- a/drivers/soc/visconti/Kconfig
> > +++ b/drivers/soc/visconti/Kconfig
> > @@ -1 +1,7 @@
> > +if ARCH_VISCONTI
> > +
> > +config VISCONTI_AFFINE
> > +    bool "Visconti Affine driver"
> > +
> > +endif
> >
> > diff --git a/drivers/soc/visconti/Makefile
> > b/drivers/soc/visconti/Makefile index 8d710da08..b25a726c3 100644
> > --- a/drivers/soc/visconti/Makefile
> > +++ b/drivers/soc/visconti/Makefile
> > @@ -4,3 +4,5 @@
> >  #
> >
> >  obj-y += ipa_common.o
> > +
> > +obj-$(CONFIG_VISCONTI_AFFINE) += affine/
> > diff --git a/drivers/soc/visconti/affine/Makefile
> > b/drivers/soc/visconti/affine/Makefile
> > new file mode 100644
> > index 000000000..82f83b2d6
> > --- /dev/null
> > +++ b/drivers/soc/visconti/affine/Makefile
> > @@ -0,0 +1,6 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# Makefile for the Visconti AFFINE driver #
> > +
> > +obj-y += affine.o hwd_affine.o
> > diff --git a/drivers/soc/visconti/affine/affine.c
> > b/drivers/soc/visconti/affine/affine.c
> > new file mode 100644
> > index 000000000..eea045dcf
> > --- /dev/null
> > +++ b/drivers/soc/visconti/affine/affine.c
> > @@ -0,0 +1,451 @@
> > +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
> > +/* Toshiba Visconti Affine Accelerator Support
> > + *
> > + * (C) Copyright 2022 TOSHIBA CORPORATION
> > + * (C) Copyright 2022 Toshiba Electronic Devices & Storage
> > +Corporation  */
> > +
> > +#include <linux/delay.h>
> > +#include <linux/init.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iommu.h>
> > +#include <linux/kernel.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/module.h>
> > +#include <linux/mutex.h>
> > +#include <linux/of_device.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/poll.h>
> > +#include <linux/wait.h>
> > +
> > +#include "hwd_affine.h"
> > +#include "../ipa_common.h"
> > +#include "../uapi/affine.h"
> > +
> > +struct affine_priv {
> > +	struct device *dev;
> > +	struct miscdevice miscdev;
> > +	struct mutex lock;
> > +	void __iomem *regs;
> > +	int irq;
> > +	wait_queue_head_t waitq;
> > +	enum drv_ipa_state status;
> > +	unsigned int hwd_event;
> > +	unsigned int poll_event;
> > +	int id;
> > +	char name[16];
> > +	bool dma_coherent;
> > +	struct hwd_affine_status hwd_status;
> > +
> > +	struct dma_buf_attachment
> *dba[DRV_AFFINE_BUFFER_INDEX_MAX];
> > +	struct sg_table *sgt[DRV_AFFINE_BUFFER_INDEX_MAX];
> > +	enum dma_data_direction
> dma_dir[DRV_AFFINE_BUFFER_INDEX_MAX];
> > +	unsigned int dma_count;
> > +
> > +	dma_addr_t buffer_iova[DRV_AFFINE_BUFFER_INDEX_MAX];
> > +};
> > +
> > +static u32 affine_ipa_addr_to_iova(struct affine_priv *priv, struct
> > +drv_ipa_addr addr) {
> > +	u32 iova = 0;
> > +
> > +	if (addr.buffer_index < priv->dma_count &&
> > +	    addr.offset < priv->dba[addr.buffer_index]->dmabuf->size)
> > +		iova = priv->buffer_iova[addr.buffer_index] + addr.offset;
> > +	return iova;
> > +}
> > +
> > +static int affine_attach_dma_buf(struct affine_priv *priv, unsigned int
> buffer_index,
> > +				 struct drv_ipa_buffer_info *buffer_info) {
> > +	int ret = 0;
> > +	dma_addr_t addr;
> > +
> > +	if (buffer_index >= DRV_AFFINE_BUFFER_INDEX_MAX) {
> > +		dev_err(priv->dev, "Buffer index invalid: index=%d\n",
> buffer_index);
> > +		return -EINVAL;
> > +	}
> > +
> > +	switch (buffer_info[buffer_index].direction) {
> > +	case DRV_IPA_DIR_NONE:
> > +		priv->dma_dir[priv->dma_count] = DMA_NONE;
> > +		break;
> > +	case DRV_IPA_DIR_TO_DEVICE:
> > +		priv->dma_dir[priv->dma_count] = DMA_TO_DEVICE;
> > +		break;
> > +	case DRV_IPA_DIR_FROM_DEVICE:
> > +		priv->dma_dir[priv->dma_count] = DMA_FROM_DEVICE;
> > +		break;
> > +	case DRV_IPA_DIR_BIDIRECTION:
> > +		priv->dma_dir[priv->dma_count] = DMA_BIDIRECTIONAL;
> > +		break;
> > +	default:
> > +		dev_err(priv->dev, "DMA direction invalid: index=%d
> dir=%d\n", buffer_index,
> > +			buffer_info[buffer_index].direction);
> > +		return -EINVAL;
> > +	}
> > +
> > +	if (!buffer_info[buffer_index].coherent) {
> > +		priv->dev->dma_coherent = false;
> > +		if (priv->dma_coherent)
> > +			priv->dma_dir[priv->dma_count] = DMA_NONE;
> > +	}
> > +
> > +	ret = ipa_attach_dmabuf(priv->dev, buffer_info[buffer_index].fd,
> > +				&priv->dba[priv->dma_count],
> &priv->sgt[priv->dma_count], &addr,
> > +				priv->dma_dir[priv->dma_count]);
> > +	if (ret == 0) {
> > +		priv->dma_count++;
> > +		priv->buffer_iova[buffer_index] = addr;
> > +	}
> > +
> > +	priv->dev->dma_coherent = priv->dma_coherent;
> > +
> > +	return ret;
> > +}
> > +
> > +static void affine_detach_dma_buf(struct affine_priv *priv) {
> > +	struct dma_buf *dmabuf;
> > +	int i;
> > +
> > +	for (i = 0; i < priv->dma_count; i++) {
> > +		dmabuf = priv->dba[i]->dmabuf;
> > +		dma_buf_unmap_attachment(priv->dba[i], priv->sgt[i],
> priv->dma_dir[i]);
> > +		dma_buf_detach(dmabuf, priv->dba[i]);
> > +		dma_buf_put(dmabuf);
> > +	}
> > +}
> > +
> > +static irqreturn_t affine_irq(int irq, void *dev_id) {
> > +	struct affine_priv *priv = dev_id;
> > +
> > +	priv->hwd_event = hwd_affine_irq_handler(priv->id);
> > +
> > +	disable_irq_nosync(priv->irq);
> > +
> > +	return IRQ_WAKE_THREAD;
> > +}
> > +
> > +static irqreturn_t affine_irq_thread(int irq, void *dev_id) {
> > +	struct affine_priv *priv = dev_id;
> > +	unsigned long delay = 1;
> > +
> > +	mutex_lock(&priv->lock);
> > +	affine_detach_dma_buf(priv);
> > +
> > +	hwd_affine_get_status(priv->id, &priv->hwd_status);
> > +
> > +	priv->status = DRV_IPA_STATE_IDLE;
> > +
> > +	/* status should be updated before poll_event so that
> > +	 * when poll() returns, user context must observe state as idle
> > +	 */
> > +	smp_wmb();
> > +
> > +	if (priv->hwd_event == HWD_AFFINE_EVENT_DONE)
> > +		priv->poll_event = IPA_POLL_EVENT_DONE;
> > +	else
> > +		priv->poll_event = IPA_POLL_EVENT_ERROR;
> > +
> > +	/* General barrier to avoid re-ordering of priv->poll_event=N and
> > +	 * waitqueue_active()
> > +	 */
> > +	smp_mb();
> > +
> > +	/* Threads going to sleep in poll() can miss wakeup, when wakeup is
> done
> > +	 * between event check in ipa_poll() and sleeping. Wakeup repeatedly.
> > +	 */
> > +	while (waitqueue_active(&priv->waitq)) {
> > +		wake_up_interruptible(&priv->waitq);
> > +
> > +		WARN_ON(delay > IPA_WAKEUP_RETRY_DELAY);
> > +		usleep_range(delay, delay + 1);
> > +		delay += delay;
> > +	}
> > +
> > +	mutex_unlock(&priv->lock);
> > +
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static void affine_start(struct affine_priv *priv, struct
> > +hwd_affine_descriptor *desc) {
> > +	hwd_affine_start(priv->id, desc);
> > +}
> > +
> > +static int affine_ioctl_start(struct affine_priv *priv, unsigned long
> > +arg) {
> > +	struct hwd_affine_descriptor hwd_desc;
> > +	struct drv_affine_descriptor desc;
> > +	int ret = 0;
> > +	int i;
> > +
> > +	ret = mutex_lock_interruptible(&priv->lock);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (priv->status == DRV_IPA_STATE_BUSY) {
> > +		dev_dbg(priv->dev, "busy: %d\n", priv->status);
> > +		ret = -EBUSY;
> > +		goto err1;
> > +	}
> > +
> > +	if (copy_from_user(&desc, (void __user *)arg, sizeof(struct
> drv_affine_descriptor))) {
> > +		dev_err(priv->dev, "Descriptor memory access error\n");
> > +		ret = -EFAULT;
> > +		goto err1;
> > +	}
> > +
> > +	if (DRV_AFFINE_BIT_CONFIG_DESC_FINAL !=
> > +	    (desc.config_done & DRV_AFFINE_BIT_CONFIG_DESC_FINAL)) {
> > +		dev_err(priv->dev, "Descriptor configuration not
> complete\n");
> > +		ret = -EINVAL;
> > +		goto err1;
> > +	}
> > +
> > +	priv->dma_count = 0;
> > +
> > +	/* setup buffer */
> > +	for (i = 0; i < desc.buffer_info_num; i++) {
> > +		ret = affine_attach_dma_buf(priv, i, desc.buffer_info);
> > +		if (ret) {
> > +			dev_err(priv->dev, "dma buf attach error:
> index=%d\n", i);
> > +			goto err2;
> > +		}
> > +		dev_dbg(priv->dev, "@buffer[%d]@: fd=%d %s iova=%llx\n", i,
> desc.buffer_info[i].fd,
> > +			desc.buffer_info[i].coherent ? "coherent" :
> "non-coherent",
> > +			(uint64_t)priv->buffer_iova[i]);
> > +	}
> > +
> > +	memcpy(&hwd_desc, &desc, sizeof(struct hwd_affine_descriptor));
> > +
> > +	hwd_desc.src_addr = affine_ipa_addr_to_iova(priv, desc.src_addr);
> > +	if (hwd_desc.src_addr == 0) {
> > +		dev_err(priv->dev, "IPA address to iova conversion error:
> src_addr %s: %d\n",
> > +			__func__, __LINE__);
> > +		ret = -EINVAL;
> > +		goto err2;
> > +	}
> > +
> > +	hwd_desc.dst_addr = affine_ipa_addr_to_iova(priv, desc.dst_addr);
> > +	if (hwd_desc.dst_addr == 0) {
> > +		dev_err(priv->dev, "IPA address to iova conversion error:
> dst_addr %s: %d\n",
> > +			__func__, __LINE__);
> > +		ret = -EINVAL;
> > +		goto err2;
> > +	}
> > +
> > +	hwd_desc.tbl_addr = 0;
> > +	if (desc.tbl_ptch != 0) {
> > +		hwd_desc.tbl_addr = affine_ipa_addr_to_iova(priv,
> desc.tbl_addr);
> > +		if (hwd_desc.tbl_addr == 0) {
> > +			dev_err(priv->dev,
> > +				"IPA address to iova conversion error:
> tbl_addr %s: %d\n", __func__,
> > +				__LINE__);
> > +			ret = -EINVAL;
> > +			goto err2;
> > +		}
> > +	}
> > +
> > +	dev_dbg(priv->dev, "src: 0x%x\n", hwd_desc.src_addr);
> > +	dev_dbg(priv->dev, "dst: 0x%x\n", hwd_desc.dst_addr);
> > +	dev_dbg(priv->dev, "tbl: 0x%x\n", hwd_desc.tbl_addr);
> > +
> > +	affine_start(priv, &hwd_desc);
> > +
> > +	priv->poll_event = IPA_POLL_EVENT_NONE;
> > +	priv->hwd_event = 0;
> > +	priv->status = DRV_IPA_STATE_BUSY;
> > +	/* Barrier to prevent affine_irq() from setting priv->hwd_event
> > +	 * before it is reset above
> > +	 */
> > +	smp_wmb();
> > +	enable_irq(priv->irq);
> > +
> > +	mutex_unlock(&priv->lock);
> > +
> > +	return ret;
> > +
> > +err2:
> > +	affine_detach_dma_buf(priv);
> > +err1:
> > +	mutex_unlock(&priv->lock);
> > +	return ret;
> > +}
> > +
> > +static int affine_ioctl_get_status(struct affine_priv *priv, unsigned
> > +long arg) {
> > +	struct drv_affine_status status;
> > +	struct hwd_affine_status hwd_status;
> > +	int ret = 0;
> > +
> > +	ret = mutex_lock_interruptible(&priv->lock);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (priv->status == DRV_IPA_STATE_BUSY)
> > +		hwd_affine_get_status(priv->id, &hwd_status);
> > +	else
> > +		hwd_status = priv->hwd_status;
> > +
> > +	status.state = priv->status;
> > +	mutex_unlock(&priv->lock);
> > +
> > +	status.comp_lack_data = hwd_status.comp_lack_data;
> > +	status.comp_over_coordinate = hwd_status.comp_over_coordinate;
> > +	status.comp_over_data = hwd_status.comp_over_data;
> > +	status.comp_size = hwd_status.comp_size;
> > +	status.comp_syntax_error = hwd_status.comp_syntax_error;
> > +	if (copy_to_user((void __user *)arg, &status, sizeof(struct
> drv_affine_status))) {
> > +		dev_err(priv->dev, "status memory access error\n");
> > +		ret = -EFAULT;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static long affine_ioctl(struct file *fp, unsigned int cmd, unsigned
> > +long arg) {
> > +	struct affine_priv *priv = container_of(fp->private_data, struct
> affine_priv, miscdev);
> > +	int ret = 0;
> > +
> > +	switch (cmd) {
> > +	case IOC_IPA_START:
> > +		ret = affine_ioctl_start(priv, arg);
> > +		break;
> > +	case IOC_IPA_GET_STATUS:
> > +		ret = affine_ioctl_get_status(priv, arg);
> > +		break;
> > +	default:
> > +		ret = -ENOIOCTLCMD;
> > +		break;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > +static __poll_t affine_poll(struct file *fp, poll_table *wait) {
> > +	struct affine_priv *priv = container_of(fp->private_data, struct
> affine_priv, miscdev);
> > +	__poll_t mask = 0;
> > +	unsigned int poll_event;
> > +
> > +	poll_wait(fp, &priv->waitq, wait);
> > +
> > +	/* Barrier to avoid re-ordering of poll_wait() and event load
> > +	 * Read barrier here and release barrier in poll_wait() together will
> > +	 * prevent re-ordering
> > +	 */
> > +	smp_rmb();
> > +	poll_event = priv->poll_event;
> > +	if (poll_event != IPA_POLL_EVENT_NONE) {
> > +		if (poll_event == IPA_POLL_EVENT_DONE)
> > +			mask = EPOLLIN | EPOLLRDNORM;
> > +		else
> > +			mask = EPOLLERR;
> > +	}
> > +	return mask;
> > +}
> > +
> > +static const struct file_operations affine_fops = {
> > +	.owner = THIS_MODULE,
> > +	.unlocked_ioctl = affine_ioctl,
> > +	.poll = affine_poll,
> > +};
> > +
> > +static int affine_probe(struct platform_device *pdev) {
> > +	struct device *dev = &pdev->dev;
> > +	struct affine_priv *priv;
> > +	int ret;
> > +
> > +	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
> > +	if (!priv)
> > +		return -ENOMEM;
> > +
> > +	mutex_init(&priv->lock);
> > +
> > +	/* update DMA mask */
> > +	priv->dma_coherent = dev->dma_coherent;
> > +	ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(36));
> > +	if (ret)
> > +		return ret;
> > +
> > +	priv->regs = devm_platform_ioremap_resource(pdev, 0);
> > +	if (IS_ERR(priv->regs))
> > +		return PTR_ERR(priv->regs);
> > +
> > +	priv->irq = platform_get_irq(pdev, 0);
> > +	if (priv->irq < 0) {
> > +		dev_err(dev, "failed to acquire irq resource\n");
> > +		return -ENOENT;
> > +	}
> > +	ret = devm_request_threaded_irq(dev, priv->irq, affine_irq,
> affine_irq_thread, 0, "affine",
> > +					priv);
> > +	if (ret) {
> > +		dev_err(dev, "irq request failed\n");
> > +		return ret;
> > +	}
> > +	disable_irq(priv->irq);
> > +
> > +	ret = of_property_read_u32(dev->of_node, "index", &priv->id);
> > +	if (ret) {
> > +		dev_err(dev, "failed to acquire irq resource\n");
> > +		return ret;
> > +	}
> > +
> > +	hwd_affine_initialize(priv->id, priv->regs);
> > +
> > +	snprintf(priv->name, sizeof(priv->name), "affine%d", priv->id);
> > +	priv->miscdev.minor = MISC_DYNAMIC_MINOR;
> > +	priv->miscdev.name = priv->name;
> > +	priv->miscdev.fops = &affine_fops;
> > +	ret = misc_register(&priv->miscdev);
> > +	if (ret) {
> > +		dev_err(dev, "misc registration failed\n");
> > +		hwd_affine_uninitialize(priv->id);
> > +		return ret;
> > +	}
> > +
> > +	priv->dev = dev;
> > +	platform_set_drvdata(pdev, priv);
> > +
> > +	init_waitqueue_head(&priv->waitq);
> > +
> > +	priv->status = DRV_IPA_STATE_IDLE;
> > +	return 0;
> > +}
> > +
> > +static int affine_remove(struct platform_device *pdev) {
> > +	struct affine_priv *priv = platform_get_drvdata(pdev);
> > +
> > +	misc_deregister(&priv->miscdev);
> > +	hwd_affine_uninitialize(priv->id);
> > +	return 0;
> > +}
> > +
> > +static const struct of_device_id affine_of_match[] = {
> > +	{
> > +		.compatible = "toshiba,visconti-affine",
> > +	},
> > +	{},
> > +};
> > +MODULE_DEVICE_TABLE(platform, affine_of_match);
> > +
> > +static struct platform_driver affine_driver = {
> > +	.probe = affine_probe,
> > +	.remove = affine_remove,
> > +	.driver = {
> > +		.name = "visconti_affine",
> > +		.of_match_table = of_match_ptr(affine_of_match),
> > +	},
> > +};
> > +module_platform_driver(affine_driver);
> > +
> > +MODULE_AUTHOR("Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>");
> > +MODULE_DESCRIPTION("Toshiba Visconti affine driver");
> > +MODULE_LICENSE("Dual BSD/GPL");
> > diff --git a/drivers/soc/visconti/affine/hwd_affine.c
> > b/drivers/soc/visconti/affine/hwd_affine.c
> > new file mode 100644
> > index 000000000..6720a0212
> > --- /dev/null
> > +++ b/drivers/soc/visconti/affine/hwd_affine.c
> > @@ -0,0 +1,206 @@
> > +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
> > +/* Toshiba Visconti Affine Accelerator Support
> > + *
> > + * (C) Copyright 2022 TOSHIBA CORPORATION
> > + * (C) Copyright 2022 Toshiba Electronic Devices & Storage
> > +Corporation  */
> > +
> > +#include <linux/delay.h>
> > +#include <linux/io.h>
> > +
> > +#include "hwd_affine.h"
> > +#include "hwd_affine_reg.h"
> > +
> > +#define HWD_AFFINE_CLEAR_BITS	  GENMASK(15, 0)
> > +#define HWD_AFFINE_INTMASK_NORMAL (0x04100)
> > +
> > +#define HWD_AFFINE_INT_END	  BIT(0)
> > +#define HWD_AFFINE_INT_TCOMP_ERR0 BIT(16) #define
> > +HWD_AFFINE_INT_TCOMP_ERR1 BIT(17) #define
> HWD_AFFINE_INT_TCOMP_ERR2
> > +BIT(18) #define HWD_AFFINE_INT_TCOMP_ERR3 BIT(19) #define
> > +HWD_AFFINE_INT_TCOMP_SIZE BIT(20)
> > +
> > +#define HWD_AFFINE_HWD_MASK_ADDR GENMASK(31, 0)
> > +
> > +#define HWD_AFFINE_START_CMD BIT(0)
> > +
> > +/**
> > + * struct hwd_affine_resources - HWD driver internal resource
> > +structure  */ struct hwd_affine_resources {
> > +	struct hwd_affine_reg *reg;
> > +	struct hwd_affine_status status;
> > +};
> > +
> > +/* HWD driver internal resource */
> > +static struct hwd_affine_resources
> > +hwd_affine_resources[HWD_AFFINE_DEVICE_MAX] = {};
> > +
> > +/**
> > + * hwd_affine_initialize() - Initialize AFFINE device
> > + *
> > + * @module_id: @ref hwd_affine_device "id" of the h/w module
> > + * @vaddr: register base virtual address  */ void
> > +hwd_affine_initialize(u32 module_id, void *vaddr) {
> > +	struct hwd_affine_resources *res =
> &hwd_affine_resources[module_id];
> > +
> > +	/* Initialize the device */
> > +	res->reg = (struct hwd_affine_reg *)vaddr; }
> > +
> > +/**
> > + * hwd_affine_uninitialize() - Uninitialize AFFINE device
> > + *
> > + * @module_id: @ref hwd_affine_device "id" of the h/w module  */ void
> > +hwd_affine_uninitialize(u32 module_id) {
> > +	struct hwd_affine_resources *res =
> &hwd_affine_resources[module_id];
> > +
> > +	/* Deinitialize the device */
> > +	res->reg = NULL;
> > +}
> > +
> > +#define NUM_AFFINE_PARAMS  6
> > +#define NUM_HOMO_PARAMS	   3
> > +#define NUM_DISTORT_PARAMS 12
> > +
> > +/**
> > + * hwd_affine_start() - Start AFFINE device
> > + *
> > + * @module_id: @ref hwd_affine_device "id" of the h/w module
> > + * @desc: Pointer to AFFINE descriptor structure  */ void
> > +hwd_affine_start(u32 module_id, const struct hwd_affine_descriptor
> > +*desc) {
> > +	struct hwd_affine_resources *res =
> &hwd_affine_resources[module_id];
> > +	struct hwd_affine_reg *reg;
> > +	int i;
> > +
> > +	reg = res->reg;
> > +
> > +	/* Initialize status */
> > +	res->status.comp_syntax_error = 0;
> > +	res->status.comp_over_coordinate = 0;
> > +	res->status.comp_over_data = 0;
> > +	res->status.comp_lack_data = 0;
> > +	res->status.comp_size = 0;
> > +
> > +	/* Clear interrupt status */
> > +	writel(HWD_AFFINE_CLEAR_BITS, &reg->statc);
> > +
> > +	/* Reset T_RST bit, C_RST bit, and O_RST bits*/
> > +	writel(desc->ctrl, &reg->ctrl);
> > +	readl(&reg->ctrl);
> > +
> > +	writel(desc->src_size, &reg->src_size);
> > +	writel((u32)(desc->src_addr & HWD_AFFINE_HWD_MASK_ADDR),
> &reg->src_addr);
> > +	writel(desc->dst_bgn, &reg->dst_bgn);
> > +	writel(desc->dst_size, &reg->dst_size);
> > +	writel((u32)(desc->dst_addr & HWD_AFFINE_HWD_MASK_ADDR),
> &reg->dst_addr);
> > +	writel(desc->l_ptch, &reg->l_ptch);
> > +
> > +	for (i = 0; i < NUM_AFFINE_PARAMS; i++)
> > +		writel(desc->affine_param[i], &reg->affine_param[i]);
> > +
> > +	for (i = 0; i < NUM_HOMO_PARAMS; i++)
> > +		writel(desc->homo_param[i], &reg->homo_param[i]);
> > +
> > +	/* AFFINE Table */
> > +	writel((u32)(desc->tbl_addr & HWD_AFFINE_HWD_MASK_ADDR),
> &reg->tbl_addr);
> > +	writel(desc->tbl_ptch, &reg->tbl_ptch);
> > +	writel(desc->tbl_comp_size, &reg->tbl_comp_size);
> > +
> > +	for (i = 0; i < NUM_DISTORT_PARAMS; i++)
> > +		writel(desc->dist_param[i], &reg->dist_param[i]);
> > +
> > +	writel(HWD_AFFINE_INTMASK_NORMAL, &reg->intm);
> > +
> > +	writel(desc->mode, &reg->mode);
> > +	writel(desc->mode2, &reg->mode2);
> > +	writel(desc->adj_p, &reg->adj_p);
> > +
> > +	/* Data Sync */
> > +	dsb(st);
> > +
> > +	/* Kick AFFINE */
> > +	writel(HWD_AFFINE_START_CMD, &reg->ctrl); }
> > +
> > +/**
> > + * hwd_affine_irq_handler() - HWD AFFINE interrupt handler
> > + *
> > + * @module_id: @ref hwd_affine_device "id" of the h/w module
> > + * Return: following bitfields
> > + * - HWD_AFFINE_EVENT_DONE: HW processing is completed
> > + * - HWD_AFFINE_EVENT_SYNTAX_ERROR: HW Table Syntax error
> > + * - HWD_AFFINE_EVENT_OVER_COORD: HW Table decoding coordinate
> error
> > + * - HWD_AFFINE_EVENT_OVER_DATA: HW Table over compressed data
> error
> > + * - HWD_AFFINE_EVENT_LACK_DATA: HW Table compressed data
> > +insufficient error
> > + * - HWD_AFFINE_EVENT_SIZE_ERROR: HW Table smaller than the size
> > +actual size of compressed data  */
> > +u32 hwd_affine_irq_handler(u32 module_id) {
> > +	u32 event = 0;
> > +	struct hwd_affine_resources *res =
> &hwd_affine_resources[module_id];
> > +	struct hwd_affine_reg *reg;
> > +	u32 cause;
> > +
> > +	reg = res->reg;
> > +
> > +	/* Read and Clear the interrupt causes */
> > +	cause = readl(&reg->statc);
> > +	writel(cause, &reg->statc);
> > +
> > +	if (cause & HWD_AFFINE_INT_END)
> > +		/* Execution End */
> > +		event |= HWD_AFFINE_EVENT_DONE;
> > +
> > +	if (cause & HWD_AFFINE_INT_TCOMP_ERR0) {
> > +		/* Syntax error */
> > +		res->status.comp_syntax_error = 1;
> > +		event |= HWD_AFFINE_EVENT_SYNTAX_ERROR;
> > +	}
> > +
> > +	if (cause & HWD_AFFINE_INT_TCOMP_ERR1) {
> > +		/* Decoding coordinate error */
> > +		res->status.comp_over_coordinate = 1;
> > +		event |= HWD_AFFINE_EVENT_OVER_COORD;
> > +	}
> > +
> > +	if (cause & HWD_AFFINE_INT_TCOMP_ERR2) {
> > +		/* Over compressed data error */
> > +		res->status.comp_over_data = 1;
> > +		event |= HWD_AFFINE_EVENT_OVER_DATA;
> > +	}
> > +
> > +	if (cause & HWD_AFFINE_INT_TCOMP_ERR3) {
> > +		/* Compressed data insufficient error */
> > +		res->status.comp_lack_data = 1;
> > +		event |= HWD_AFFINE_EVENT_LACK_DATA;
> > +	}
> > +
> > +	if (cause & HWD_AFFINE_INT_TCOMP_SIZE) {
> > +		/* Smaller than the actual size of compressed data */
> > +		res->status.comp_size = 1;
> > +		event |= HWD_AFFINE_EVENT_SIZE_ERROR;
> > +	}
> > +
> > +	return event;
> > +}
> > +
> > +/**
> > + * hwd_affine_get_status() - HWD AFFINE Get Status
> > + *
> > + * @module_id: @ref hwd_affine_device "id" of the h/w module
> > + * @status: Pointer to status structure
> > + * Return: RETURN_OK operation completed successfully  */ void
> > +hwd_affine_get_status(u32 module_id, struct hwd_affine_status
> > +*status) {
> > +	const struct hwd_affine_resources *res =
> > +&hwd_affine_resources[module_id];
> > +
> > +	*status = res->status;
> > +}
> > diff --git a/drivers/soc/visconti/affine/hwd_affine.h
> > b/drivers/soc/visconti/affine/hwd_affine.h
> > new file mode 100644
> > index 000000000..ed17b1116
> > --- /dev/null
> > +++ b/drivers/soc/visconti/affine/hwd_affine.h
> > @@ -0,0 +1,83 @@
> > +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
> > +/* Toshiba Visconti Affine Accelerator Support
> > + *
> > + * (C) Copyright 2022 TOSHIBA CORPORATION
> > + * (C) Copyright 2022 Toshiba Electronic Devices & Storage
> > +Corporation  */
> > +
> > +#ifndef HWD_AFFINE_H
> > +#define HWD_AFFINE_H
> > +
> > +/**
> > + * enum hwd_affine_device_id - AFFINE HWD Device ID  */ enum
> > +hwd_affine_device_id {
> > +	HWD_AFFINE_DEVICE_0 = 0,
> > +	HWD_AFFINE_DEVICE_1 = 1,
> > +	HWD_AFFINE_DEVICE_MAX = 2,
> > +};
> > +
> > +/* hwd_affine_event_flags - Macro for Affine driver events */
> > +#define HWD_AFFINE_EVENT_DONE	      BIT(0)
> > +#define HWD_AFFINE_EVENT_SYNTAX_ERROR BIT(1)
> > +#define HWD_AFFINE_EVENT_OVER_COORD   BIT(2)
> > +#define HWD_AFFINE_EVENT_OVER_DATA    BIT(3)
> > +#define HWD_AFFINE_EVENT_LACK_DATA    BIT(4)
> > +#define HWD_AFFINE_EVENT_SIZE_ERROR   BIT(5)
> > +#define HWD_AFFINE_EVENT_MAX	      BIT(6)
> > +
> > +/* hwd_affine_param_num_flags Macro for Affine parameter number */
> > +#define HWD_AFFINE_AFFINE_PARAM_NUM	 (6U)
> > +#define HWD_AFFINE_UNDIST_PARAM_NUM	 (12U)
> > +#define HWD_AFFINE_HOMO_PARAM_NUM	 (3U)
> > +#define HWD_AFFINE_TEMP_UNDIST_PARAM_NUM (10U)
> > +
> > +/**
> > + * struct hwd_affine_status - HWD Affine status
> > + * @comp_size: Setting size is smaller than actual size
> > + * @comp_lack_data: Compressed table is lacking enough data for
> > +processing
> > + * @comp_over_data: Compressed table is exceeding the limit of data
> > +that can be processed
> > + * @comp_over_coordinate: Number of coordinates in compressed table
> > +data exceeded the limit
> > + * @comp_syntax_error: Syntax error of compressed table has occurred
> > + * @reserved: Padding
> > + */
> > +struct hwd_affine_status {
> > +	u32 comp_size : 1;
> > +	u32 comp_lack_data : 1;
> > +	u32 comp_over_data : 1;
> > +	u32 comp_over_coordinate : 1;
> > +	u32 comp_syntax_error : 1;
> > +	u32 reserved : 27;
> > +};
> > +
> > +/**
> > + * struct hwd_affine_descriptor - HWD AFFINE Descriptor  */ struct
> > +hwd_affine_descriptor {
> > +	u32 ctrl;
> > +	u32 mode2;
> > +	u32 mode;
> > +	u32 adj_p;
> > +	u32 src_size;
> > +	u32 src_addr;
> > +	u32 dst_bgn;
> > +	u32 dst_size;
> > +	u32 dst_addr;
> > +	u32 l_ptch;
> > +	u32 tbl_addr;
> > +	u32 tbl_ptch;
> > +	u32 affine_param[HWD_AFFINE_AFFINE_PARAM_NUM];
> > +	u32 dist_param[HWD_AFFINE_UNDIST_PARAM_NUM];
> > +	u32 homo_param[HWD_AFFINE_HOMO_PARAM_NUM];
> > +	u32 tbl_comp_size;
> > +	float temp_dist_p[HWD_AFFINE_TEMP_UNDIST_PARAM_NUM];
> > +	u16 config_done;
> > +};
> > +
> > +void hwd_affine_initialize(u32 module_id, void *vaddr); void
> > +hwd_affine_uninitialize(u32 module_id); void hwd_affine_start(u32
> > +module_id, const struct hwd_affine_descriptor *desc);
> > +u32 hwd_affine_irq_handler(u32 module_id); void
> > +hwd_affine_get_status(u32 module_id, struct hwd_affine_status
> > +*status);
> > +
> > +#endif /* HWD_AFFINE_H */
> > diff --git a/drivers/soc/visconti/affine/hwd_affine_reg.h
> > b/drivers/soc/visconti/affine/hwd_affine_reg.h
> > new file mode 100644
> > index 000000000..407d898fa
> > --- /dev/null
> > +++ b/drivers/soc/visconti/affine/hwd_affine_reg.h
> > @@ -0,0 +1,45 @@
> > +/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
> > +/* Toshiba Visconti Affine Accelerator Support
> > + *
> > + * (C) Copyright 2022 TOSHIBA CORPORATION
> > + * (C) Copyright 2022 Toshiba Electronic Devices & Storage
> > +Corporation  */
> > +
> > +#ifndef HWD_AFFINE_REG_H
> > +#define HWD_AFFINE_REG_H
> > +
> > +/* hwd_affine_params_num_flags - Macro for Affine parameter number */
> > +#define HWD_AFFINE_AFFINE_PARAMS_NUM (6) #define
> > +HWD_AFFINE_UNDIST_PARAMS_NUM (12)
> > +#define HWD_AFFINE_HOMO_PARAMS_NUM   (3)
> > +#define HWD_AFFINE_RESERVED	     (472)
> > +
> > +/**
> > + * struct hwd_affine_reg - Structure of Affine register descriptor
> > +*/ struct hwd_affine_reg {
> > +	u32 ctrl;
> > +	u32 reserved0[2];
> > +	u32 mode2;
> > +	u32 mode;
> > +	u32 adj_p;
> > +	u32 intm;
> > +	u32 statc;
> > +	u32 out_lpos;
> > +	u32 src_size;
> > +	u32 src_addr;
> > +	u32 dst_bgn;
> > +	u32 dst_size;
> > +	u32 dst_addr;
> > +	u32 l_ptch;
> > +	u32 tbl_addr;
> > +	u32 tbl_ptch;
> > +	u32 affine_param[HWD_AFFINE_AFFINE_PARAMS_NUM];
> > +	u32 dist_param[HWD_AFFINE_UNDIST_PARAMS_NUM];
> > +	u32 reserved1;
> > +	u32 homo_param[HWD_AFFINE_HOMO_PARAMS_NUM];
> > +	u32 tbl_comp_size;
> > +	u32 reserved2[HWD_AFFINE_RESERVED];
> > +};
> > +
> > +#endif /* HWD_AFFINE_REG_H */
> > diff --git a/drivers/soc/visconti/uapi/affine.h
> > b/drivers/soc/visconti/uapi/affine.h
> > new file mode 100644
> > index 000000000..539261483
> > --- /dev/null
> > +++ b/drivers/soc/visconti/uapi/affine.h
> > @@ -0,0 +1,87 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * (C) Copyright 2020 Toshiba Electronic Devices & Storage
> > +Corporation  */
> > +
> > +#ifndef _UAPI_LINUX_AFIINE_H
> 
> Typo: AFIINE -> AFFINE.
> 
> Same comment as for the DNN driver: there is no documentation whatsoever
> about what it does or what data it expects.
> 
> Some high-level documentation is sorely needed.
> 
> Regards,
> 
> 	Hans
> 
> > +#define _UAPI_LINUX_AFIINE_H
> > +
> > +#include <linux/ioctl.h>
> > +#include <linux/types.h>
> > +#include "ipa.h"
> > +
> > +#define DRV_AFFINE_BIT_CONFIG_DESC_FINAL (0x8000U)
> > +#define DRV_AFFINE_BUFFER_INDEX_MAX	 (3)
> > +
> > +#define DRV_AFFINE_AFFINE_PARAM_NUM	 (6U)
> > +#define DRV_AFFINE_UNDIST_PARAM_NUM	 (12U)
> > +#define DRV_AFFINE_HOMO_PARAM_NUM	 (3U)
> > +#define DRV_AFFINE_TEMP_UNDIST_PARAM_NUM (10U)
> > +
> > +/**
> > + * struct drv_affine_status - AFFINE IPA status for IOC_IPA_GET_STATUS
> > + * @state:                State of driver
> > + * @comp_size:            Setting size is smaller than actual size
> > + * @comp_lack_data:       Compressed table is lacking enough data for
> processing
> > + * @comp_over_data:       Compressed table is exceeding the limit of
> data that can be processed
> > + * @comp_over_coordinate: Number of coordinates in compressed table
> data exceeded the limit
> > + * @comp_syntax_error:    Syntax error of compressed table has occurred
> > + * @reserved:             Padding
> > + */
> > +struct drv_affine_status {
> > +	enum drv_ipa_state state;
> > +	uint32_t comp_size : 1;
> > +	uint32_t comp_lack_data : 1;
> > +	uint32_t comp_over_data : 1;
> > +	uint32_t comp_over_coordinate : 1;
> > +	uint32_t comp_syntax_error : 1;
> > +	uint32_t reserved : 27;
> > +};
> > +
> > +/**
> > + * struct drv_affine_descriptor - AFFINE IPA descriptor for IOC_IPA_START
> > + * @ctrl:            Operation control of AFFINE
> > + * @mode2:           Operation mode for the AFFINE execution
> > + * @mode:            Operation mode for the AFFINE execution
> > + * @adj_p:           Output Pixel Value Adjustment
> > + * @src_size:        Input image size
> > + * @src_addr:        Start address of input image
> > + * @dst_bgn:         Start coordinate of the output rectangle area
> > + * @dst_size:        Size of the output rectangle area
> > + * @dst_addr:        Address of the output rectangle area
> > + * @l_ptch:          Line pitch of both the input image and output
> rectangle area
> > + * @tbl_addr:        Start address of the table used in the table conversion
> mode.
> > + * @tbl_ptch:        Line pitch of the table used in the table conversion
> mode
> > + * @affine_param:    Parameters for Affine transformation
> > + * @dist_param:      Parameters for Distortion correction
> > + * @homo_param:      Parameters Homography transformation
> > + * @tbl_comp_size:   Table compression size
> > + * @temp_dist_p:     Temporary buffer to hold distortion input parameters
> > + * @config_done:     Flags of called configuration
> > + * @buffer_info:     Table of buffer information
> > + * @buffer_info_num: Number of buffer_info  */ struct
> > +drv_affine_descriptor {
> > +	uint32_t ctrl;
> > +	uint32_t mode2;
> > +	uint32_t mode;
> > +	uint32_t adj_p;
> > +	uint32_t src_size;
> > +	struct drv_ipa_addr src_addr;
> > +	uint32_t dst_bgn;
> > +	uint32_t dst_size;
> > +	struct drv_ipa_addr dst_addr;
> > +	uint32_t l_ptch;
> > +	struct drv_ipa_addr tbl_addr;
> > +	uint32_t tbl_ptch;
> > +	uint32_t affine_param[DRV_AFFINE_AFFINE_PARAM_NUM];
> > +	uint32_t dist_param[DRV_AFFINE_UNDIST_PARAM_NUM];
> > +	uint32_t homo_param[DRV_AFFINE_HOMO_PARAM_NUM];
> > +	uint32_t tbl_comp_size;
> > +	float temp_dist_p[DRV_AFFINE_TEMP_UNDIST_PARAM_NUM];
> > +	uint16_t config_done;
> > +	struct drv_ipa_buffer_info
> buffer_info[DRV_AFFINE_BUFFER_INDEX_MAX];
> > +	int32_t buffer_info_num;
> > +};
> > +
> > +#endif /* _UAPI_LINUX_AFIINE_H */

[PATCH v2 4/4] MAINTAINERS: Add entries for Toshiba Visconti AFFINE image processing accelerator

[Yuji Ishikawa]

Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
 MAINTAINERS | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index dd36acc87..231b2c6f9 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2796,12 +2796,14 @@ F:	Documentation/devicetree/bindings/net/toshiba,visconti-dwmac.yaml
 F:	Documentation/devicetree/bindings/gpio/toshiba,gpio-visconti.yaml
 F:	Documentation/devicetree/bindings/pci/toshiba,visconti-pcie.yaml
 F:	Documentation/devicetree/bindings/pinctrl/toshiba,visconti-pinctrl.yaml
+F:  Documentation/devicetree/bindings/soc/visconti/
 F:	Documentation/devicetree/bindings/watchdog/toshiba,visconti-wdt.yaml
 F:	arch/arm64/boot/dts/toshiba/
 F:	drivers/net/ethernet/stmicro/stmmac/dwmac-visconti.c
 F:	drivers/gpio/gpio-visconti.c
 F:	drivers/pci/controller/dwc/pcie-visconti.c
 F:	drivers/pinctrl/visconti/
+F:	drivers/soc/visconti/
 F:	drivers/watchdog/visconti_wdt.c
 N:	visconti
 
-- 
2.17.1

Re: [PATCH v2 0/4] Add Toshiba Visconti AFFINE image processing accelerator driver

[Laurent Pinchart]

Hi Yuji,

Thank you for the patch. It's nice to see contributions from Toshiba in
the image acceleration domain :-)

I'll start with a high-level question before diving into detailed
review. Why is this implemented in drivers/soc/ with a custom userspace
API, and not as a V4L2 memory-to-memory driver ?

On Wed, Apr 27, 2022 at 10:23:41PM +0900, Yuji Ishikawa wrote:
> This series is the AFFINE image processing accelerator driver for Toshiba's ARM SoC, Visconti[0].
> This provides DT binding documentation, device driver, MAINTAINER files.
> 
> The second patch "soc: visconti: Add Toshiba Visconti image processing accelerator common source"
> is commonly used among acclerator drivers (affine, dnn, dspif, pyramid).
> 
> Best regards,
> Yuji
> 
> [0]: https://toshiba.semicon-storage.com/ap-en/semiconductor/product/image-recognition-processors-visconti.html
>   
>   dt-bindings: soc: visconti: Add Toshiba Visconti AFFINE image
>     v1 -> v2:
>       - No update
> 
>   soc: visconti: Add Toshiba Visconti image processing accelerator common source
>     v1 -> v2:
>       - apply checkpatch.pl --strict
>   
>   soc: visconti: Add Toshiba Visconti AFFINE image processing accelerator
>     v1 -> v2:
>       - apply checkpatch.pl --strict
>       - rename hwd_AFFINE_xxxx to hwd_affine_xxxx
> 
>   MAINTAINERS: Add entries for Toshiba Visconti AFFINE image processing accelerator
>     v1 -> v2:
>       - No update
> 
> Change in V2:
>   - apply checkpatch.pl --strict
>   - rename hwd_AFFINE_xxxx to hwd_affine_xxxx
> 
> Yuji Ishikawa (4):
>   dt-bindings: soc: visconti: Add Toshiba Visconti AFFINE image
>     processing accelerator bindings
>   soc: visconti: Add Toshiba Visconti image processing accelerator
>     common source
>   soc: visconti: Add Toshiba Visconti AFFINE image processing
>     accelerator
>   MAINTAINERS: Add entries for Toshiba Visconti AFFINE image processing
>     accelerator
> 
>  .../soc/visconti/toshiba,visconti-affine.yaml |  53 ++
>  MAINTAINERS                                   |   2 +
>  drivers/soc/Kconfig                           |   1 +
>  drivers/soc/Makefile                          |   1 +
>  drivers/soc/visconti/Kconfig                  |   7 +
>  drivers/soc/visconti/Makefile                 |   8 +
>  drivers/soc/visconti/affine/Makefile          |   6 +
>  drivers/soc/visconti/affine/affine.c          | 451 ++++++++++++++++++
>  drivers/soc/visconti/affine/hwd_affine.c      | 206 ++++++++
>  drivers/soc/visconti/affine/hwd_affine.h      |  83 ++++
>  drivers/soc/visconti/affine/hwd_affine_reg.h  |  45 ++
>  drivers/soc/visconti/ipa_common.c             |  55 +++
>  drivers/soc/visconti/ipa_common.h             |  18 +
>  drivers/soc/visconti/uapi/affine.h            |  87 ++++
>  drivers/soc/visconti/uapi/ipa.h               |  88 ++++
>  15 files changed, 1111 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/soc/visconti/toshiba,visconti-affine.yaml
>  create mode 100644 drivers/soc/visconti/Kconfig
>  create mode 100644 drivers/soc/visconti/Makefile
>  create mode 100644 drivers/soc/visconti/affine/Makefile
>  create mode 100644 drivers/soc/visconti/affine/affine.c
>  create mode 100644 drivers/soc/visconti/affine/hwd_affine.c
>  create mode 100644 drivers/soc/visconti/affine/hwd_affine.h
>  create mode 100644 drivers/soc/visconti/affine/hwd_affine_reg.h
>  create mode 100644 drivers/soc/visconti/ipa_common.c
>  create mode 100644 drivers/soc/visconti/ipa_common.h
>  create mode 100644 drivers/soc/visconti/uapi/affine.h
>  create mode 100644 drivers/soc/visconti/uapi/ipa.h

-- 
Regards,

Laurent Pinchart

RE: [PATCH v2 0/4] Add Toshiba Visconti AFFINE image processing accelerator driver

[yuji2.ishikawa]

Hi Laurent,

Thank you for your comment.

We had never imagined that affine driver can be a V4L2 driver.
Affine is one of the accelerators in Visconti, and some accelerators receive/yield non-picture data.
Also, as the original accelerator drivers were implemented for kernel 4.19.x, we were not aware of the latest V4L2 architecture.
Currently, we assume accelerator drivers are kicked individually, not in an image processing pipeline, therefore simple misc driver is enough solution.

Is there any memory-to-memory driver sample/skelton to bring me better understanding?

Best regards,
Yuji

> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Sent: Thursday, April 28, 2022 7:04 AM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>
> Cc: Rob Herring <robh+dt@kernel.org>; iwamatsu nobuhiro(岩松 信洋 □ＳＷ
> Ｃ◯ＡＣＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>; Sumit Semwal
> <sumit.semwal@linaro.org>; Christian König <christian.koenig@amd.com>;
> linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> linux-media@vger.kernel.org; dri-devel@lists.freedesktop.org;
> linaro-mm-sig@lists.linaro.org
> Subject: Re: [PATCH v2 0/4] Add Toshiba Visconti AFFINE image processing
> accelerator driver
> 
> Hi Yuji,
> 
> Thank you for the patch. It's nice to see contributions from Toshiba in the image
> acceleration domain :-)
> 
> I'll start with a high-level question before diving into detailed review. Why is
> this implemented in drivers/soc/ with a custom userspace API, and not as a
> V4L2 memory-to-memory driver ?
> 
> On Wed, Apr 27, 2022 at 10:23:41PM +0900, Yuji Ishikawa wrote:
> > This series is the AFFINE image processing accelerator driver for Toshiba's
> ARM SoC, Visconti[0].
> > This provides DT binding documentation, device driver, MAINTAINER files.
> >
> > The second patch "soc: visconti: Add Toshiba Visconti image processing
> accelerator common source"
> > is commonly used among acclerator drivers (affine, dnn, dspif, pyramid).
> >
> > Best regards,
> > Yuji
> >
> > [0]:
> >
> https://toshiba.semicon-storage.com/ap-en/semiconductor/product/image-
> > recognition-processors-visconti.html
> >
> >   dt-bindings: soc: visconti: Add Toshiba Visconti AFFINE image
> >     v1 -> v2:
> >       - No update
> >
> >   soc: visconti: Add Toshiba Visconti image processing accelerator common
> source
> >     v1 -> v2:
> >       - apply checkpatch.pl --strict
> >
> >   soc: visconti: Add Toshiba Visconti AFFINE image processing accelerator
> >     v1 -> v2:
> >       - apply checkpatch.pl --strict
> >       - rename hwd_AFFINE_xxxx to hwd_affine_xxxx
> >
> >   MAINTAINERS: Add entries for Toshiba Visconti AFFINE image processing
> accelerator
> >     v1 -> v2:
> >       - No update
> >
> > Change in V2:
> >   - apply checkpatch.pl --strict
> >   - rename hwd_AFFINE_xxxx to hwd_affine_xxxx
> >
> > Yuji Ishikawa (4):
> >   dt-bindings: soc: visconti: Add Toshiba Visconti AFFINE image
> >     processing accelerator bindings
> >   soc: visconti: Add Toshiba Visconti image processing accelerator
> >     common source
> >   soc: visconti: Add Toshiba Visconti AFFINE image processing
> >     accelerator
> >   MAINTAINERS: Add entries for Toshiba Visconti AFFINE image processing
> >     accelerator
> >
> >  .../soc/visconti/toshiba,visconti-affine.yaml |  53 ++
> >  MAINTAINERS                                   |   2 +
> >  drivers/soc/Kconfig                           |   1 +
> >  drivers/soc/Makefile                          |   1 +
> >  drivers/soc/visconti/Kconfig                  |   7 +
> >  drivers/soc/visconti/Makefile                 |   8 +
> >  drivers/soc/visconti/affine/Makefile          |   6 +
> >  drivers/soc/visconti/affine/affine.c          | 451
> ++++++++++++++++++
> >  drivers/soc/visconti/affine/hwd_affine.c      | 206 ++++++++
> >  drivers/soc/visconti/affine/hwd_affine.h      |  83 ++++
> >  drivers/soc/visconti/affine/hwd_affine_reg.h  |  45 ++
> >  drivers/soc/visconti/ipa_common.c             |  55 +++
> >  drivers/soc/visconti/ipa_common.h             |  18 +
> >  drivers/soc/visconti/uapi/affine.h            |  87 ++++
> >  drivers/soc/visconti/uapi/ipa.h               |  88 ++++
> >  15 files changed, 1111 insertions(+)
> >  create mode 100644
> > Documentation/devicetree/bindings/soc/visconti/toshiba,visconti-affine
> > .yaml  create mode 100644 drivers/soc/visconti/Kconfig  create mode
> > 100644 drivers/soc/visconti/Makefile  create mode 100644
> > drivers/soc/visconti/affine/Makefile
> >  create mode 100644 drivers/soc/visconti/affine/affine.c
> >  create mode 100644 drivers/soc/visconti/affine/hwd_affine.c
> >  create mode 100644 drivers/soc/visconti/affine/hwd_affine.h
> >  create mode 100644 drivers/soc/visconti/affine/hwd_affine_reg.h
> >  create mode 100644 drivers/soc/visconti/ipa_common.c  create mode
> > 100644 drivers/soc/visconti/ipa_common.h  create mode 100644
> > drivers/soc/visconti/uapi/affine.h
> >  create mode 100644 drivers/soc/visconti/uapi/ipa.h
> 
> --
> Regards,
> 
> Laurent Pinchart