[PATCH RFC 00/12] remoteproc: add support for any virtio device

[PATCH RFC 00/12] remoteproc: add support for any virtio device

[Francesco Valla]

Hello,

this patch series introduces the possibility to support generic virtio
devices over the remotepoc transport, whereas today only rpmsg and
virtio-console are supported.

== Introduction ==

Support for devices other than rpmsg was originally planned [1] and is
declared inside the documentation [2], but is in practice not there for
the majority of (if not all) the platforms that provide remoteproc
capabilities due to memory allocation.

While vrings are pre-allocated in an area that is reachable by both the
local (i.e.; Linux) and remote processors, buffers produced by virtio
drivers aren't, since they typically get allocated through kmalloc.
The aforementioned rpmsg and virtio-console drivers instead use a trick
to overcome this limitation and allocate these buffers directly from
the remoteproc device's coherent memory area, somewhat breaking the
separation between the driver and the underlying transport.

== Well, nice, but why? ==

Main usecase is sharing/virtualization of devices in hypervisor-less
mixed-criticality contexts, where a subset of peripherals are controlled
by a "safety" real-time processor but still need to be used by the Linux
world. Several solutions have been / are being proposed [3] [4], but
none of them re-uses the existing, standardized virtio specifications. 

== The proposal ==

The proposed approach is to introduce a bounce buffering mechanism that
is transparent to the drivers and can expose to remoteproc devices only
memory areas they can access. This is obtained by defining the .map
memeber of each registered vdev and use the map() and unmap() callback
to bounce data to and from the remote processor, just like the swiotlb
framework is doing in other contexts, using the device's coherent memory
area and the associated functions to allocate the bounce buffers.

During the map() callback the address of the incoming buffer is compared
against the coherent memory address base and size, to pass through
buffers already suitable for remoteproc usage (e.g.: the ones allocated
by the rpmsg framework).

== Status and open points ==

The series was tested against a custom Zephyr application [5] running on
the Cortex-M33 processor of an i.MX93 and exposing six different virtio
devices:

  - rpmsg
  - entropy (rng)
  - gpio
  - i2c
  - spi
  - can

On top of three of them (unsurprisingly: i2c, spi, and gpio) several
devices where declared inside Linux devicetree and successfully used
(well, technically I'm still experiencing difficulties with gpio
interrupts not firing on the M33, but that's not really related to the
series). 

Several open points are still present, and needs to be either
investigated or discussed:

  - for each bounce buffer an entire page is allocated from the coherent
    memory pool; this is a waste for most of the allocations, which take
    on average 32 to 64 bytes. An option can be to initialize a DMA pool
    on one page and allocate small buffers from it?

  - the support in its current form allocates more memory than before
    (for bounce buffer tracking) also for existing usecases (i.e.,
    mainly rpmsg).

  - an additional issue still exist - and is not solved by this series -
    for a subset of virtio devices: communication through the device's
    config space. The remoteproc transport expects this config space to
    be somewhat constant, and there is no provision to sync changes made
    by the driver with the remote device. This prevents e.g.
    virtio-input to work.  

  - device de-registration on remoteproc stop is causing oopses (under
    investigation - might no be strictly tied to the series)

== Patches breakdown ==

Patches 1 and 2 are cleanups to the remoteproc-virtio driver and could
be applied independently of this series.

Patch 3 was submitted a couple of months ago [6] and paves the road for
the actual support of generic virtio devices, removing the fixed number
of 2 for the vrings associated to a vdev.

Patch 4 introduces two new APIs for coherent memory areas associated to
devices that are used later.

Patch 5 might somewhat be controversial, as it unconditionally defines
the VIRTIO_F_VERSION_1 feature for all vdevs. This is required to
support some virtio device types, and there is no other mean of
defining it, since the field reserved for features inside the resource
table is limited to 32 bits. Given that the 1.x virtio specifications
are ~10 years old this still seems reasonable.

Patch 6 is were the bounce buffering mechanism is introduced; another
feature (VIRTIO_F_ACCESS_PLATFORM) is there unconditionally defined to
force the virtio framework to use the new map APIs.

Patches 7 and 8 are new devicetree bindings, the first for spi-virtio
(modelled against the existing ones for gpio-virtio and i2c-virtio) and
the second for declaring virtio device inside a devicetree. This is not
required for some devices (e.g.: can, net, gpu), but for others is
necessary to declare child devices and link them.

Patch 9 is used to convince the remoteproc-virtio transport to parse the
bindings just defined; it is worth noting that the virtio framework
already has the support for devicetree declarations and this adds only
the glue between the existing support and remoteproc.

Patches 10 and 11 are i.MX-specific and enable the usage of the newly
introduced support on this family of platforms. The first one might
probably be sumbitted as-is independently of the series, as it aligns
the behavior of imx-rproc to the other platforms in relation to mailbox
usage.

Finally, patch 12 is the PoC that has been used to develop and test the
series and shall not be merged.

======

Thank you in advance for any comment you may want to leave.

Regards,
Francesco

[1] https://lore.kernel.org/all/1330589497-4139-1-git-send-email-ohad@wizery.com/
[2] https://elixir.bootlin.com/linux/v7.2.5/source/Documentation/staging/remoteproc.rst#L26
[3] https://lore.kernel.org/linux-remoteproc/20260721204704.400781-1-shenwei.wang@oss.nxp.com/
[4] https://cfp.embedded-recipes.org/er2026/talk/PCYPJP/
[5] https://github.com/WallaceIT/zephyr/tree/multi_vdev
[6] https://lore.kernel.org/all/20260621-vring_flex-v1-1-c6c582fbe94b@valla.it/

Signed-off-by: Francesco Valla <francesco@valla.it>
---
Francesco Valla (12):
      remoteproc: virtio: cleanup rproc_add_virtio_dev error path
      remoteproc: virtio: replace commas with semicolons
      remoteproc: virtio: support dynamic number of vrings
      dma-coherent: add base and size APIs
      remoteproc: always report VIRTIO_F_VERSION_1 feature
      remoteproc: virtio: add bounce buffering for data buffers
      dt-bindings: spi: add bindings for spi-virtio
      dt-bindings: remoteproc: add remoteproc-virtio
      remoteproc: search for a fwnode during vdev registration
      remoteproc: imx_rproc: always use non-blocking mailboxes
      dt-bindings: remoteproc: imx-rproc: support virtio
      PoC: arm64: dts: imx93-11x11-frdm: add multiple vdevs

 .../bindings/remoteproc/fsl,imx-rproc.yaml         |   3 +-
 .../bindings/remoteproc/remoteproc-virtio.yaml     |  89 +++++++++
 .../devicetree/bindings/spi/spi-virtio.yaml        |  52 +++++
 arch/arm64/boot/dts/freescale/imx93-11x11-frdm.dts | 128 +++++++++++-
 drivers/remoteproc/imx_rproc.c                     |  49 +----
 drivers/remoteproc/imx_rproc.h                     |   1 -
 drivers/remoteproc/remoteproc_core.c               |  43 +++-
 drivers/remoteproc/remoteproc_virtio.c             | 222 ++++++++++++++++++---
 include/linux/dma-map-ops.h                        |  10 +
 include/linux/remoteproc.h                         |  24 ++-
 kernel/dma/coherent.c                              |  34 ++++
 11 files changed, 562 insertions(+), 93 deletions(-)
---
base-commit: 9b87fdc9af2fbfcdb5c24a64139685ef80f6573f
change-id: 20260915-remoteproc_virtio_map-bcf32a5fab54

Best regards,
--  
Francesco Valla <francesco@valla.it>

[PATCH RFC 01/12] remoteproc: virtio: cleanup rproc_add_virtio_dev error path

[Francesco Valla]

There is no cleanup to perform in case an error occurs inside
rproc_add_virtio_dev, so remove the goto label and just return in such
cases.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 drivers/remoteproc/remoteproc_virtio.c | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index d5e9ff045a28..096bcba527c1 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -376,9 +376,8 @@ static int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id)
 	int ret;
 
 	if (rproc->ops->kick == NULL) {
-		ret = -EINVAL;
 		dev_err(dev, ".kick method not defined for %s\n", rproc->name);
-		goto out;
+		return -EINVAL;
 	}
 
 	/* Try to find dedicated vdev buffer carveout */
@@ -394,7 +393,7 @@ static int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id)
 								 mem->of_resm_idx);
 			if (ret) {
 				dev_err(dev, "Can't associate reserved memory\n");
-				goto out;
+				return ret;
 			}
 		} else {
 			if (mem->va) {
@@ -412,7 +411,7 @@ static int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id)
 							   mem->len);
 			if (ret < 0) {
 				dev_err(dev, "Failed to associate buffer\n");
-				goto out;
+				return ret;
 			}
 		}
 	} else {
@@ -431,11 +430,10 @@ static int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id)
 
 	/* Allocate virtio device */
 	vdev = kzalloc_obj(*vdev);
-	if (!vdev) {
-		ret = -ENOMEM;
-		goto out;
-	}
-	vdev->id.device	= id,
+	if (!vdev)
+		return -ENOMEM;
+
+	vdev->id.device = id,
 	vdev->config = &rproc_virtio_config_ops,
 	vdev->dev.parent = dev;
 	vdev->dev.release = rproc_virtio_dev_release;
@@ -447,13 +445,12 @@ static int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id)
 	if (ret) {
 		put_device(&vdev->dev);
 		dev_err(dev, "failed to register vdev: %d\n", ret);
-		goto out;
+		return ret;
 	}
 
 	dev_info(dev, "registered %s (type %d)\n", dev_name(&vdev->dev), id);
 
-out:
-	return ret;
+	return 0;
 }
 
 /**

-- 
2.55.0

[PATCH RFC 02/12] remoteproc: virtio: replace commas with semicolons

[Francesco Valla]

While not technically wrong, the comma at the end of an assignment
directive can lead to nasty bugs. Just replace the two occurrences with
semicolons.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 drivers/remoteproc/remoteproc_virtio.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 096bcba527c1..4dacd93bf238 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -433,8 +433,8 @@ static int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id)
 	if (!vdev)
 		return -ENOMEM;
 
-	vdev->id.device = id,
-	vdev->config = &rproc_virtio_config_ops,
+	vdev->id.device = id;
+	vdev->config = &rproc_virtio_config_ops;
 	vdev->dev.parent = dev;
 	vdev->dev.release = rproc_virtio_dev_release;
 

-- 
2.55.0

[PATCH RFC 03/12] remoteproc: virtio: support dynamic number of vrings

[Francesco Valla]

The number of vrings for each vdev has been fixed to 2 since the
introduction of multi-vdev support [1]; this is completely fine for the
rpmsg usecase, but can conflict with other virtio devices (CAN for
example requires 3 virtqueues, entropy only 1, network a variable number
and so on).

Remove the static vring allocation, transforming it to a flex array that
is allocated at vdev probe time; for the existent usecases (i.e.: mainly
rpmsg) this leads to no functional change, except the additional memory
used for the counter associated to the new array.

The maximum number of virtqueues is limited to 256 due to the uint8_t
value used inside the resource table to indicate the number of vring to
allocate; for this reason, no additional plausibility check is performed
on the number of vrings indicated by the resource table.

As a side effect, this also fixes the single virtqueue usecase, which
was apparently supported also before but for which the remove action
caused an error (because the remove action was trying to unmap also the
second vring, which was in fact not mapped).

[1] https://lore.kernel.org/all/1330589497-4139-5-git-send-email-ohad@wizery.com/

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 drivers/remoteproc/remoteproc_core.c   |  7 -------
 drivers/remoteproc/remoteproc_virtio.c | 21 +++++++++++++--------
 include/linux/remoteproc.h             | 10 ++++------
 3 files changed, 17 insertions(+), 21 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 1ed406714849..b7d9957534bf 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -473,7 +473,6 @@ static int rproc_handle_vdev(struct rproc *rproc, void *ptr,
 {
 	struct fw_rsc_vdev *rsc = ptr;
 	struct device *dev = &rproc->dev;
-	struct rproc_vdev *rvdev;
 	size_t rsc_size;
 	struct rproc_vdev_data rvdev_data;
 	struct platform_device *pdev;
@@ -494,12 +493,6 @@ static int rproc_handle_vdev(struct rproc *rproc, void *ptr,
 	dev_dbg(dev, "vdev rsc: id %d, dfeatures 0x%x, cfg len %d, %d vrings\n",
 		rsc->id, rsc->dfeatures, rsc->config_len, rsc->num_of_vrings);
 
-	/* we currently support only two vrings per rvdev */
-	if (rsc->num_of_vrings > ARRAY_SIZE(rvdev->vring)) {
-		dev_err(dev, "too many vrings: %d\n", rsc->num_of_vrings);
-		return -EINVAL;
-	}
-
 	rvdev_data.id = rsc->id;
 	rvdev_data.index = rproc->nb_vdev++;
 	rvdev_data.rsc_offset = offset;
diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 4dacd93bf238..74e0da970f1d 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -115,8 +115,7 @@ static struct virtqueue *rp_find_vq(struct virtio_device *vdev,
 	void *addr;
 	int num, size;
 
-	/* we're temporarily limited to two virtqueues per rvdev */
-	if (id >= ARRAY_SIZE(rvdev->vring))
+	if (id >= rvdev->num_vrings)
 		return ERR_PTR(-EINVAL);
 
 	if (!name)
@@ -500,17 +499,20 @@ static int rproc_virtio_probe(struct platform_device *pdev)
 	if (!rvdev_data)
 		return -EINVAL;
 
-	rvdev = devm_kzalloc(dev, sizeof(*rvdev), GFP_KERNEL);
+	rsc = rvdev_data->rsc;
+
+	rvdev = kzalloc_flex(*rvdev, vring, rsc->num_of_vrings);
 	if (!rvdev)
 		return -ENOMEM;
 
 	rvdev->id = rvdev_data->id;
 	rvdev->rproc = rproc;
 	rvdev->index = rvdev_data->index;
+	rvdev->num_vrings = rsc->num_of_vrings;
 
 	ret = copy_dma_range_map(dev, rproc->dev.parent);
 	if (ret)
-		return ret;
+		goto free_rvdev;
 
 	/* Make device dma capable by inheriting from parent's capabilities */
 	set_dma_ops(dev, get_dma_ops(rproc->dev.parent));
@@ -524,13 +526,11 @@ static int rproc_virtio_probe(struct platform_device *pdev)
 	platform_set_drvdata(pdev, rvdev);
 	rvdev->pdev = pdev;
 
-	rsc = rvdev_data->rsc;
-
 	/* parse the vrings */
 	for (i = 0; i < rsc->num_of_vrings; i++) {
 		ret = rproc_parse_vring(rvdev, rsc, i);
 		if (ret)
-			return ret;
+			goto free_rvdev;
 	}
 
 	/* remember the resource offset*/
@@ -566,6 +566,9 @@ static int rproc_virtio_probe(struct platform_device *pdev)
 	for (i--; i >= 0; i--)
 		rproc_free_vring(&rvdev->vring[i]);
 
+free_rvdev:
+	kfree(rvdev);
+
 	return ret;
 }
 
@@ -576,7 +579,7 @@ static void rproc_virtio_remove(struct platform_device *pdev)
 	struct rproc_vring *rvring;
 	int id;
 
-	for (id = 0; id < ARRAY_SIZE(rvdev->vring); id++) {
+	for (id = 0; id < rvdev->num_vrings; id++) {
 		rvring = &rvdev->vring[id];
 		rproc_free_vring(rvring);
 	}
@@ -585,6 +588,8 @@ static void rproc_virtio_remove(struct platform_device *pdev)
 	rproc_remove_rvdev(rvdev);
 
 	put_device(&rproc->dev);
+
+	kfree(rvdev);
 }
 
 /* Platform driver */
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index a44368737b39..c3ba51fe9e54 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -319,10 +319,6 @@ struct rproc_subdev {
 	void (*unprepare)(struct rproc_subdev *subdev);
 };
 
-/* we currently support only two vrings per rvdev */
-
-#define RVDEV_NUM_VRINGS 2
-
 /**
  * struct rproc_vring - remoteproc vring state
  * @va:	virtual address
@@ -350,9 +346,10 @@ struct rproc_vring {
  * @id: virtio device id (as in virtio_ids.h)
  * @node: list node
  * @rproc: the rproc handle
- * @vring: the vrings for this vdev
  * @rsc_offset: offset of the vdev's resource entry
  * @index: vdev position versus other vdev declared in resource table
+ * @num_vrings: the number of vrings for this vdev
+ * @vring: the vrings for this vdev
  */
 struct rproc_vdev {
 	struct rproc_subdev subdev;
@@ -361,9 +358,10 @@ struct rproc_vdev {
 	unsigned int id;
 	struct list_head node;
 	struct rproc *rproc;
-	struct rproc_vring vring[RVDEV_NUM_VRINGS];
 	u32 rsc_offset;
 	u32 index;
+	unsigned int num_vrings;
+	struct rproc_vring vring[] __counted_by(num_vrings);
 };
 
 struct rproc *rproc_get_by_phandle(phandle phandle);

-- 
2.55.0

[PATCH RFC 04/12] dma-coherent: add base and size APIs

[Francesco Valla]

Add two new APIs for the dma coherent memory to get the base and size of
the pool associated to a device.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 include/linux/dma-map-ops.h | 10 ++++++++++
 kernel/dma/coherent.c       | 34 ++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)

diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
index 8fae2b7deb20..580b69321d56 100644
--- a/include/linux/dma-map-ops.h
+++ b/include/linux/dma-map-ops.h
@@ -156,6 +156,8 @@ int dma_alloc_from_dev_coherent(struct device *dev, ssize_t size,
 int dma_release_from_dev_coherent(struct device *dev, int order, void *vaddr);
 int dma_mmap_from_dev_coherent(struct device *dev, struct vm_area_struct *vma,
 		void *cpu_addr, size_t size, int *ret);
+size_t dma_dev_coherent_size(struct device *dev);
+dma_addr_t dma_dev_coherent_base(struct device *dev);
 #else
 static inline int dma_declare_coherent_memory(struct device *dev,
 		phys_addr_t phys_addr, dma_addr_t device_addr, size_t size)
@@ -167,6 +169,14 @@ static inline int dma_declare_coherent_memory(struct device *dev,
 #define dma_release_from_dev_coherent(dev, order, vaddr) (0)
 #define dma_mmap_from_dev_coherent(dev, vma, vaddr, order, ret) (0)
 static inline void dma_release_coherent_memory(struct device *dev) { }
+static inline size_t dma_dev_coherent_size(struct device *dev)
+{
+	return 0;
+}
+static inline dma_addr_t dma_dev_coherent_base(struct device *dev)
+{
+	return DMA_MAPPING_ERROR;
+}
 #endif /* CONFIG_DMA_DECLARE_COHERENT */
 
 #ifdef CONFIG_DMA_GLOBAL_POOL
diff --git a/kernel/dma/coherent.c b/kernel/dma/coherent.c
index 45bbae947f4b..f4aa5813604b 100644
--- a/kernel/dma/coherent.c
+++ b/kernel/dma/coherent.c
@@ -277,6 +277,40 @@ int dma_mmap_from_dev_coherent(struct device *dev, struct vm_area_struct *vma,
 	return __dma_mmap_from_coherent(mem, vma, vaddr, size, ret);
 }
 
+/**
+ * dma_dev_coherent_size() - total size of the device coherent pool
+ * @dev:	device from which we allocate memory
+ *
+ * Returns the total size of the coherent memory pool associated to the given
+ * device, or 0 in case of no pool.
+ */
+size_t dma_dev_coherent_size(struct device *dev)
+{
+	struct dma_coherent_mem *mem = dev_get_coherent_memory(dev);
+
+	if (!mem)
+		return 0;
+
+	return (mem->size << PAGE_SHIFT);
+}
+
+/**
+ * dma_dev_coherent_base() - base address of the device coherent pool
+ * @dev:	device from which we allocate memory
+ *
+ * Returns the base address of the coherent memory pool associated with the
+ * device, or DMA_MAPPING_ERROR in case of no pool.
+ */
+dma_addr_t dma_dev_coherent_base(struct device *dev)
+{
+	struct dma_coherent_mem *mem = dev_get_coherent_memory(dev);
+
+	if (!mem)
+		return DMA_MAPPING_ERROR;
+
+	return dma_get_device_base(dev, mem);
+}
+
 #ifdef CONFIG_DMA_GLOBAL_POOL
 static struct dma_coherent_mem *dma_coherent_default_memory __ro_after_init;
 

-- 
2.55.0

[PATCH RFC 05/12] remoteproc: always report VIRTIO_F_VERSION_1 feature

[Francesco Valla]

There is currently no way to report the VIRTIO_F_VERSION_1 using the
resource table alone, as the per-vdev feature array is limited to 32
bits.

Considering that the VirtIO 1.0 specification is now ~10 years old,
always report the VIRTIO_F_VERSION_1 feature, as some drivers depend on
it.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 drivers/remoteproc/remoteproc_virtio.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index 74e0da970f1d..cfd66d9d1c9e 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -249,7 +249,7 @@ static u64 rproc_virtio_get_features(struct virtio_device *vdev)
 
 	rsc = (void *)rvdev->rproc->table_ptr + rvdev->rsc_offset;
 
-	return rsc->dfeatures;
+	return rsc->dfeatures | (1ULL << VIRTIO_F_VERSION_1);
 }
 
 static void rproc_transport_features(struct virtio_device *vdev)
@@ -275,14 +275,16 @@ static int rproc_virtio_finalize_features(struct virtio_device *vdev)
 	/* Give virtio_rproc a chance to accept features. */
 	rproc_transport_features(vdev);
 
-	/* Make sure we don't have any features > 32 bits! */
-	BUG_ON((u32)vdev->features != vdev->features);
+	/* Make sure we don't have any features > 32 bits except VIRTIO_F_VERSION_1 */
+	if (WARN_ON_ONCE((u32)vdev->features !=
+			 (vdev->features & ~(1ULL << VIRTIO_F_VERSION_1))))
+		return -1;
 
 	/*
 	 * Remember the finalized features of our vdev, and provide it
 	 * to the remote processor once it is powered on.
 	 */
-	rsc->gfeatures = vdev->features;
+	rsc->gfeatures = vdev->features & ~(1ULL << VIRTIO_F_VERSION_1);
 
 	return 0;
 }

-- 
2.55.0

[PATCH RFC 06/12] remoteproc: virtio: add bounce buffering for data buffers

[Francesco Valla]

Depending on the driver originating them, data buffers used for virtio
communication can either:

  - already be allocated from the coherent memory area that is
    accessible by the remote processor; this is the case of rpmsg
    and the rproc flavor of virtio-console;
  - be allocated from generic kmem, and thus not accessible directly by
    the remote processor.

Exploiting the map operations, which are used by the virtio framework
when VIRTIO_F_ACCESS_PLATFORM is part of a vdev's feature flags, add
bounce buffering for the second case: when the map() callback is called
for a buffer, one or more pages of coherent memory are allocated and
data is copied to them, then they are exposed to the remote processor;
the data is  then bounced back on unmap().

The first case is not impacted, since buffers already suitable for
remote transmission are passed through.

With the bounce buffering in place, any kind of virtio device can be
supported through the remoteproc-virtio transport, at least from a
data exchange standpoint.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 drivers/remoteproc/remoteproc_virtio.c | 182 +++++++++++++++++++++++++++++++--
 include/linux/remoteproc.h             |  14 +++
 2 files changed, 190 insertions(+), 6 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_virtio.c b/drivers/remoteproc/remoteproc_virtio.c
index cfd66d9d1c9e..d21b3b8044df 100644
--- a/drivers/remoteproc/remoteproc_virtio.c
+++ b/drivers/remoteproc/remoteproc_virtio.c
@@ -241,7 +241,14 @@ static void rproc_virtio_reset(struct virtio_device *vdev)
 	dev_dbg(&vdev->dev, "reset !\n");
 }
 
-/* provide the vdev features as retrieved from the firmware */
+/* Provide the vdev features as retrieved from the firmware, plus the following
+ * additional ones:
+ *   - VIRTIO_F_VERSION_1 that is required by some non-rpmsg virtio devices
+ *   - VIRTIO_F_ACCESS_PLATFORM to force usage of the map operations
+ */
+#define RPROC_VIRTIO_STATIC_FEATURES \
+	((1ULL << VIRTIO_F_VERSION_1) | (1ULL << VIRTIO_F_ACCESS_PLATFORM))
+
 static u64 rproc_virtio_get_features(struct virtio_device *vdev)
 {
 	struct rproc_vdev *rvdev = vdev_to_rvdev(vdev);
@@ -249,7 +256,7 @@ static u64 rproc_virtio_get_features(struct virtio_device *vdev)
 
 	rsc = (void *)rvdev->rproc->table_ptr + rvdev->rsc_offset;
 
-	return rsc->dfeatures | (1ULL << VIRTIO_F_VERSION_1);
+	return rsc->dfeatures | RPROC_VIRTIO_STATIC_FEATURES;
 }
 
 static void rproc_transport_features(struct virtio_device *vdev)
@@ -275,16 +282,16 @@ static int rproc_virtio_finalize_features(struct virtio_device *vdev)
 	/* Give virtio_rproc a chance to accept features. */
 	rproc_transport_features(vdev);
 
-	/* Make sure we don't have any features > 32 bits except VIRTIO_F_VERSION_1 */
+	/* Make sure we don't have any features > 32 bits */
 	if (WARN_ON_ONCE((u32)vdev->features !=
-			 (vdev->features & ~(1ULL << VIRTIO_F_VERSION_1))))
+			 (vdev->features & ~RPROC_VIRTIO_STATIC_FEATURES)))
 		return -1;
 
 	/*
 	 * Remember the finalized features of our vdev, and provide it
 	 * to the remote processor once it is powered on.
 	 */
-	rsc->gfeatures = vdev->features & ~(1ULL << VIRTIO_F_VERSION_1);
+	rsc->gfeatures = vdev->features & ~RPROC_VIRTIO_STATIC_FEATURES;
 
 	return 0;
 }
@@ -337,6 +344,151 @@ static const struct virtio_config_ops rproc_virtio_config_ops = {
 	.set		= rproc_virtio_set,
 };
 
+static inline unsigned int rproc_virtio_bounce_slot(struct device *dma_dev,
+						    dma_addr_t dma_handle)
+{
+	const dma_addr_t dma_base = dma_dev_coherent_base(dma_dev);
+
+	return (dma_handle - dma_base) >> PAGE_SHIFT;
+}
+
+static dma_addr_t rproc_virtio_map_page(union virtio_map map, struct page *page,
+					unsigned long offset, size_t size,
+					enum dma_data_direction dir,
+					unsigned long attrs)
+{
+	struct device *dev = map.dma_dev;
+	struct rproc_vdev *rvdev = dev_get_drvdata(dev);
+	dma_addr_t dma_base = dma_dev_coherent_base(dev);
+	size_t dma_size = dma_dev_coherent_size(dev);
+	phys_addr_t paddr = page_to_phys(page) + offset;
+	void *vaddr = page_to_virt(page) + offset;
+	struct rproc_map_record *record;
+	dma_addr_t map_handle;
+	void *bounce;
+
+	// No need to allocate a bounce buffer if the memory to map is already
+	// part of the device's coherent pool.
+	if (paddr >= dma_base && paddr < (dma_base + dma_size)) {
+		// The allocation details will be recorded also in this case,
+		// indicating that no bounce buffer was allocated.
+		map_handle = (dma_addr_t)paddr;
+		bounce = NULL;
+	} else {
+		// Allocate bounce buffer from device coherent memory
+		bounce = dma_alloc_coherent(dev, size, &map_handle, GFP_KERNEL | __GFP_ZERO);
+		if (!bounce)
+			return DMA_MAPPING_ERROR;
+
+		// Copy data to bounce buffer
+		memcpy(bounce, vaddr, size);
+	}
+
+	// Save bounce details
+	record = &rvdev->map_records[rproc_virtio_bounce_slot(dev, map_handle)];
+
+	record->original = vaddr;
+	record->size = size;
+	record->bounce = bounce;
+
+	return map_handle;
+}
+
+static void rproc_virtio_unmap_page(union virtio_map map, dma_addr_t map_handle,
+				    size_t size, enum dma_data_direction dir,
+				    unsigned long attrs)
+{
+	struct device *dev = map.dma_dev;
+	struct rproc_vdev *rvdev = dev_get_drvdata(dev);
+	unsigned int slot = rproc_virtio_bounce_slot(dev, map_handle);
+	struct rproc_map_record *record = &rvdev->map_records[slot];
+
+	WARN_ON(size != record->size);
+
+	// If a bounce buffer was used, copy data back to original one
+	if (record->bounce) {
+		memcpy(record->original, record->bounce, record->size);
+
+		dma_free_coherent(dev, record->size, record->bounce, map_handle);
+	}
+
+	record->original = NULL;
+	record->size = 0;
+	record->bounce = NULL;
+}
+
+static void rproc_virtio_sync_single_for_cpu(union virtio_map map,
+					     dma_addr_t map_handle,
+					     size_t size,
+					     enum dma_data_direction dir)
+{
+	struct device *dev = map.dma_dev;
+
+	dma_sync_single_range_for_cpu(dev, (map_handle & PAGE_MASK),
+				      offset_in_page(map_handle), size, dir);
+}
+
+static void rproc_virtio_sync_single_for_device(union virtio_map map,
+						dma_addr_t map_handle,
+						size_t size,
+						enum dma_data_direction dir)
+{
+	struct device *dev = map.dma_dev;
+
+	dma_sync_single_range_for_device(dev, (map_handle & PAGE_MASK),
+					 offset_in_page(map_handle), size, dir);
+}
+
+static void *rproc_virtio_alloc(union virtio_map map, size_t size,
+				dma_addr_t *map_handle, gfp_t gfp)
+{
+	struct device *dev = map.dma_dev;
+
+	return dma_alloc_coherent(dev, size, map_handle, gfp);
+}
+
+static void rproc_virtio_free(union virtio_map map, size_t size, void *vaddr,
+			      dma_addr_t map_handle, unsigned long attrs)
+{
+	struct device *dev = map.dma_dev;
+
+	dma_free_coherent(dev, size, vaddr, map_handle);
+}
+
+static bool rproc_virtio_need_sync(union virtio_map map, dma_addr_t map_handle)
+{
+	struct device *dev = map.dma_dev;
+
+	return dma_need_sync(dev, map_handle);
+}
+
+static int rproc_virtio_mapping_error(union virtio_map map, dma_addr_t map_handle)
+{
+	if (unlikely(map_handle == DMA_MAPPING_ERROR))
+		return -ENOMEM;
+
+	return 0;
+}
+
+static inline size_t rproc_virtio_max_mapping_size(union virtio_map map)
+{
+	struct device *dev = map.dma_dev;
+
+	return dma_dev_coherent_size(dev);
+}
+
+static const struct virtio_map_ops rproc_virtio_map_ops = {
+	.map_page = rproc_virtio_map_page,
+	.unmap_page = rproc_virtio_unmap_page,
+	.sync_single_for_cpu = rproc_virtio_sync_single_for_cpu,
+	.sync_single_for_device = rproc_virtio_sync_single_for_device,
+	.alloc = rproc_virtio_alloc,
+	.free = rproc_virtio_free,
+	.need_sync = rproc_virtio_need_sync,
+	.mapping_error = rproc_virtio_mapping_error,
+	.max_mapping_size = rproc_virtio_max_mapping_size,
+};
+
 /*
  * This function is called whenever vdev is released, and is responsible
  * to decrement the remote processor's refcount which was taken when vdev was
@@ -355,6 +507,8 @@ static void rproc_virtio_dev_release(struct device *dev)
 	of_reserved_mem_device_release(&rvdev->pdev->dev);
 	dma_release_coherent_memory(&rvdev->pdev->dev);
 
+	kvfree(rvdev->map_records);
+
 	put_device(&rvdev->pdev->dev);
 }
 
@@ -429,13 +583,29 @@ static int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id)
 		of_reserved_mem_device_init_by_idx(dev, np, 0);
 	}
 
+	/* Allocate one tracking record for each page of the device reserved
+	 * memory. Contiguous memory is not required for this array, which can
+	 * also be quite big (depending on the size of the coherent memory), so
+	 * let's use vmalloc for this allocation.
+	 */
+	rvdev->map_records = kvcalloc(dma_dev_coherent_size(dev) >> PAGE_SHIFT,
+				      sizeof(*rvdev->map_records),
+				      GFP_KERNEL);
+	if (!rvdev->map_records) {
+		dev_err(dev, "failed to allocate memory for map records\n");
+		return -ENOMEM;
+	}
+
 	/* Allocate virtio device */
 	vdev = kzalloc_obj(*vdev);
-	if (!vdev)
+	if (!vdev) {
+		kvfree(rvdev->map_records);
 		return -ENOMEM;
+	}
 
 	vdev->id.device = id;
 	vdev->config = &rproc_virtio_config_ops;
+	vdev->map = &rproc_virtio_map_ops;
 	vdev->dev.parent = dev;
 	vdev->dev.release = rproc_virtio_dev_release;
 
diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h
index c3ba51fe9e54..2ff48b505ac0 100644
--- a/include/linux/remoteproc.h
+++ b/include/linux/remoteproc.h
@@ -339,10 +339,23 @@ struct rproc_vring {
 	struct virtqueue *vq;
 };
 
+/**
+ * struct rproc_map_record - remoteproc map record
+ * @original:	original virtual address
+ * @num: allocation size
+ * @bounce: bounce buffer virtual address (NULL if not used)
+ */
+struct rproc_map_record {
+	void *original;
+	size_t size;
+	void *bounce;
+};
+
 /**
  * struct rproc_vdev - remoteproc state for a supported virtio device
  * @subdev: handle for registering the vdev as a rproc subdevice
  * @pdev: remoteproc virtio platform device
+ * @map_records: array of map records
  * @id: virtio device id (as in virtio_ids.h)
  * @node: list node
  * @rproc: the rproc handle
@@ -358,6 +371,7 @@ struct rproc_vdev {
 	unsigned int id;
 	struct list_head node;
 	struct rproc *rproc;
+	struct rproc_map_record *map_records;
 	u32 rsc_offset;
 	u32 index;
 	unsigned int num_vrings;

-- 
2.55.0

[PATCH RFC 07/12] dt-bindings: spi: add bindings for spi-virtio

[Francesco Valla]

Add bindings for spi-virtio, based on virtio-device.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 .../devicetree/bindings/spi/spi-virtio.yaml        | 52 ++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/Documentation/devicetree/bindings/spi/spi-virtio.yaml b/Documentation/devicetree/bindings/spi/spi-virtio.yaml
new file mode 100644
index 000000000000..1a0a0763bd2f
--- /dev/null
+++ b/Documentation/devicetree/bindings/spi/spi-virtio.yaml
@@ -0,0 +1,52 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/i2c/i2c-virtio.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Virtio SPI Controller
+
+maintainers:
+  - Francesco Valla <francesco@valla.it>
+
+allOf:
+  - $ref: /schemas/spi/spi-controller.yaml#
+  - $ref: /schemas/virtio/virtio-device.yaml#
+
+description:
+  Virtio SPI device, see /schemas/virtio/virtio-device.yaml for more details.
+
+properties:
+  $nodename:
+    const: spi
+
+  compatible:
+    const: virtio,device2d
+
+required:
+  - compatible
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    virtio@3000 {
+        compatible = "virtio,mmio";
+        reg = <0x3000 0x100>;
+        interrupts = <41>;
+
+        spi {
+            compatible = "virtio,device2d";
+
+            #address-cells = <1>;
+            #size-cells = <0>;
+
+            sram@0 {
+                compatible = "microchip,mchp23k256";
+                reg = <0>;
+                spi-max-frequency = <20000000>;
+            };
+        };
+    };
+
+...

-- 
2.55.0

[PATCH RFC 08/12] dt-bindings: remoteproc: add remoteproc-virtio

[Francesco Valla]

Add a new binding to describe remoteproc-provided virtio devices; while
these are discovered through a resource table parsed by the remoteproc
infrastructure at runtime, their description can be needed to probe
non-discoverable buses (such as I2C) or to link consumers and suppliers.

Each vdev is described by a dedicated "group" node, which then includes
a virtio-device node, which binding is already existent and used by
virtio-mmio. Each vdev shall be stattically linked to a "group" node
using its index inside the resource table as the reg property of the
node; this permits to have multiple instances of the same type of
device.

The binding is intended to be generic and adopted by any remoteproc
provider.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 .../bindings/remoteproc/remoteproc-virtio.yaml     | 89 ++++++++++++++++++++++
 1 file changed, 89 insertions(+)

diff --git a/Documentation/devicetree/bindings/remoteproc/remoteproc-virtio.yaml b/Documentation/devicetree/bindings/remoteproc/remoteproc-virtio.yaml
new file mode 100644
index 000000000000..c4a0d84b1460
--- /dev/null
+++ b/Documentation/devicetree/bindings/remoteproc/remoteproc-virtio.yaml
@@ -0,0 +1,89 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/remoteproc/remoteproc-virtio.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Virtio devices over remoteproc
+
+description: |
+  Virtio devices ("vdevs") can be exposed using the remoteproc infrastructure
+  and its resource table. For some of them, a device tree node might be needed
+  to describe remote undiscoverable hardware and/or connect consumers and
+  providers.
+
+maintainers:
+  - Francesco Valla <francesco@valla.it>
+
+properties:
+  virtio:
+    description: Contains a group of Virtio devices exposed by the remoteproc.
+
+    properties:
+      '#address-cells':
+        const: 1
+
+      '#size-cells':
+        const: 0
+
+    patternProperties:
+      "^vdev@[0-9a-f]+$":
+        type: object
+
+        properties:
+          reg:
+            description: Virtio device index inside the resource table.
+            maxItems: 1
+
+        additionalProperties:
+          type: object
+          $ref: /schemas/virtio/virtio-device.yaml
+          maxItems: 1
+
+        required:
+          - reg
+
+    additionalProperties: false
+
+    required:
+      - '#address-cells'
+      - '#size-cells'
+
+additionalProperties: true
+
+examples:
+  - |
+    remoteproc-cm33 {
+      virtio {
+          #address-cells = <1>;
+          #size-cells = <0>;
+
+          vdev@0 {
+              reg = <0>;
+
+              gpio {
+                  compatible = "virtio,device29";
+
+                  gpio-controller;
+                  #gpio-cells = <2>;
+              };
+          };
+
+          vdev@1 {
+              reg = <1>;
+
+              i2c {
+                  compatible = "virtio,device22";
+
+                  #address-cells = <1>;
+                  #size-cells = <0>;
+
+                  eeprom@50 {
+                      compatible = "atmel,24c1025";
+                      reg = <0x50>;
+                  };
+              };
+          };
+      };
+    };
+...

-- 
2.55.0

[PATCH RFC 09/12] remoteproc: search for a fwnode during vdev registration

[Francesco Valla]

The virtio subsystem already supports a fwnode describing a vdev; search
for a specific node during vdev registration and supply it to the
platform device that is being registered.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 drivers/remoteproc/remoteproc_core.c | 36 ++++++++++++++++++++++++++++++++++--
 1 file changed, 34 insertions(+), 2 deletions(-)

diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index b7d9957534bf..e3ecc61cc1de 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -440,6 +440,29 @@ void rproc_remove_rvdev(struct rproc_vdev *rvdev)
 	if (rvdev)
 		list_del(&rvdev->node);
 }
+
+static struct fwnode_handle *rproc_get_vdev_fwnode(struct device *dev, int idx)
+{
+	struct fwnode_handle *group, *child;
+	u32 reg;
+	int ret;
+
+	group = device_get_named_child_node(dev, "virtio");
+
+	fwnode_for_each_child_node(group, child) {
+		ret = fwnode_property_read_u32(child, "reg", &reg);
+		if (ret)
+			continue;
+
+		if (idx == reg)
+			break;
+	}
+
+	fwnode_handle_put(group);
+
+	return child;
+}
+
 /**
  * rproc_handle_vdev() - handle a vdev fw resource
  * @rproc: the remote processor
@@ -475,6 +498,7 @@ static int rproc_handle_vdev(struct rproc *rproc, void *ptr,
 	struct device *dev = &rproc->dev;
 	size_t rsc_size;
 	struct rproc_vdev_data rvdev_data;
+	struct platform_device_info pdev_info;
 	struct platform_device *pdev;
 
 	/* make sure resource isn't truncated */
@@ -504,8 +528,16 @@ static int rproc_handle_vdev(struct rproc *rproc, void *ptr,
 	 * as device id, then we get duplication in sysfs, so need to use
 	 * PLATFORM_DEVID_AUTO to auto select device id.
 	 */
-	pdev = platform_device_register_data(dev, "rproc-virtio", PLATFORM_DEVID_AUTO, &rvdev_data,
-					     sizeof(rvdev_data));
+	memset(&pdev_info, 0, sizeof(pdev_info));
+	pdev_info.parent = dev;
+	pdev_info.fwnode = rproc_get_vdev_fwnode(dev->parent, rvdev_data.index);
+	pdev_info.of_node_reused = true;
+	pdev_info.name = "rproc-virtio";
+	pdev_info.id = PLATFORM_DEVID_AUTO;
+	pdev_info.data = &rvdev_data;
+	pdev_info.size_data = sizeof(rvdev_data);
+
+	pdev = platform_device_register_full(&pdev_info);
 	if (IS_ERR(pdev)) {
 		dev_err(dev, "failed to create rproc-virtio device\n");
 		return PTR_ERR(pdev);

-- 
2.55.0

[PATCH RFC 10/12] remoteproc: imx_rproc: always use non-blocking mailboxes

[Francesco Valla]

Blocking mailboxes may lead to sleep during the kick operation, which
conflicts with some virtio devices (e.g.: virtio-can) that kicks their
virtqueues with spinlock(s) held.

Align to the behavior of all the other remoteproc drivers that use a
mailbox to perform the kick (i.e.: omap, st, stm32, ti_k3_*, xlnx_r5)
and move to a non-blocking mailbox here as well.

As a side effect of this modification, remove the close-reopen action on
the mailboxes during restart and shutdown.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 drivers/remoteproc/imx_rproc.c | 49 ++++++------------------------------------
 drivers/remoteproc/imx_rproc.h |  1 -
 2 files changed, 6 insertions(+), 44 deletions(-)

diff --git a/drivers/remoteproc/imx_rproc.c b/drivers/remoteproc/imx_rproc.c
index 745ce52cd822..100440328f92 100644
--- a/drivers/remoteproc/imx_rproc.c
+++ b/drivers/remoteproc/imx_rproc.c
@@ -97,7 +97,7 @@ struct imx_rproc_mem {
 /* Linux has permission to handle the Logical Machine of remote cores */
 #define IMX_RPROC_FLAGS_SM_LMM_CTRL	BIT(0)
 
-static int imx_rproc_xtr_mbox_init(struct rproc *rproc, bool tx_block);
+static int imx_rproc_xtr_mbox_init(struct rproc *rproc);
 static void imx_rproc_free_mbox(void *data);
 
 /* Forward declarations for platform operations */
@@ -444,7 +444,7 @@ static int imx_rproc_start(struct rproc *rproc)
 	struct device *dev = priv->dev;
 	int ret;
 
-	ret = imx_rproc_xtr_mbox_init(rproc, true);
+	ret = imx_rproc_xtr_mbox_init(rproc);
 	if (ret)
 		return ret;
 
@@ -712,7 +712,7 @@ static void imx_rproc_kick(struct rproc *rproc, int vqid)
 
 static int imx_rproc_attach(struct rproc *rproc)
 {
-	return imx_rproc_xtr_mbox_init(rproc, true);
+	return imx_rproc_xtr_mbox_init(rproc);
 }
 
 static int imx_rproc_scu_api_detach(struct rproc *rproc)
@@ -877,7 +877,7 @@ static void imx_rproc_rx_callback(struct mbox_client *cl, void *msg)
 	queue_work(priv->workqueue, &priv->rproc_work);
 }
 
-static int imx_rproc_xtr_mbox_init(struct rproc *rproc, bool tx_block)
+static int imx_rproc_xtr_mbox_init(struct rproc *rproc)
 {
 	struct imx_rproc *priv = rproc->priv;
 	struct device *dev = priv->dev;
@@ -900,7 +900,7 @@ static int imx_rproc_xtr_mbox_init(struct rproc *rproc, bool tx_block)
 
 	cl = &priv->cl;
 	cl->dev = dev;
-	cl->tx_block = tx_block;
+	cl->tx_block = false;
 	cl->tx_tout = 100;
 	cl->knows_txdone = false;
 	cl->rx_callback = imx_rproc_rx_callback;
@@ -1223,22 +1223,6 @@ static int imx_rproc_detect_mode(struct imx_rproc *priv)
 	return priv->ops->detect_mode(priv->rproc);
 }
 
-static int imx_rproc_sys_off_handler(struct sys_off_data *data)
-{
-	struct rproc *rproc = data->cb_data;
-	int ret;
-
-	imx_rproc_free_mbox(rproc);
-
-	ret = imx_rproc_xtr_mbox_init(rproc, false);
-	if (ret) {
-		dev_err(&rproc->dev, "Failed to request non-blocking mbox\n");
-		return NOTIFY_BAD;
-	}
-
-	return NOTIFY_DONE;
-}
-
 static void imx_rproc_destroy_workqueue(void *data)
 {
 	struct workqueue_struct *workqueue = data;
@@ -1285,7 +1269,7 @@ static int imx_rproc_probe(struct platform_device *pdev)
 
 	INIT_WORK(&priv->rproc_work, imx_rproc_vq_work);
 
-	ret = imx_rproc_xtr_mbox_init(rproc, true);
+	ret = imx_rproc_xtr_mbox_init(rproc);
 	if (ret)
 		return ret;
 
@@ -1315,26 +1299,6 @@ static int imx_rproc_probe(struct platform_device *pdev)
 	if (rproc->state != RPROC_DETACHED)
 		rproc->auto_boot = of_property_read_bool(np, "fsl,auto-boot");
 
-	if (dcfg->flags & IMX_RPROC_NEED_SYSTEM_OFF) {
-		/*
-		 * setup mailbox to non-blocking mode in
-		 * [SYS_OFF_MODE_POWER_OFF_PREPARE, SYS_OFF_MODE_RESTART_PREPARE]
-		 * phase before invoking [SYS_OFF_MODE_POWER_OFF, SYS_OFF_MODE_RESTART]
-		 * atomic chain, see kernel/reboot.c.
-		 */
-		ret = devm_register_sys_off_handler(dev, SYS_OFF_MODE_POWER_OFF_PREPARE,
-						    SYS_OFF_PRIO_DEFAULT,
-						    imx_rproc_sys_off_handler, rproc);
-		if (ret)
-			return dev_err_probe(dev, ret, "register power off handler failure\n");
-
-		ret = devm_register_sys_off_handler(dev, SYS_OFF_MODE_RESTART_PREPARE,
-						    SYS_OFF_PRIO_DEFAULT,
-						    imx_rproc_sys_off_handler, rproc);
-		if (ret)
-			return dev_err_probe(dev, ret, "register restart handler failure\n");
-	}
-
 	pm_runtime_enable(dev);
 	ret = pm_runtime_resume_and_get(dev);
 	if (ret)
@@ -1447,7 +1411,6 @@ static const struct imx_rproc_dcfg imx_rproc_cfg_imx8ulp = {
 static const struct imx_rproc_dcfg imx_rproc_cfg_imx7ulp = {
 	.att		= imx_rproc_att_imx7ulp,
 	.att_size	= ARRAY_SIZE(imx_rproc_att_imx7ulp),
-	.flags		= IMX_RPROC_NEED_SYSTEM_OFF,
 };
 
 static const struct imx_rproc_dcfg imx_rproc_cfg_imx7d = {
diff --git a/drivers/remoteproc/imx_rproc.h b/drivers/remoteproc/imx_rproc.h
index 0d7d48352a10..9a1400d0d081 100644
--- a/drivers/remoteproc/imx_rproc.h
+++ b/drivers/remoteproc/imx_rproc.h
@@ -16,7 +16,6 @@ struct imx_rproc_att {
 };
 
 /* dcfg flags */
-#define IMX_RPROC_NEED_SYSTEM_OFF	BIT(0)
 #define IMX_RPROC_NEED_CLKS		BIT(1)
 
 struct imx_rproc_plat_ops {

-- 
2.55.0

[PATCH RFC 11/12] dt-bindings: remoteproc: imx-rproc: support virtio

[Francesco Valla]

Add support for virtio devices description.

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml b/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
index c18f71b64889..32fc415c7028 100644
--- a/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
+++ b/Documentation/devicetree/bindings/remoteproc/fsl,imx-rproc.yaml
@@ -92,6 +92,7 @@ required:
   - compatible
 
 allOf:
+  - $ref: remoteproc-virtio.yaml#
   - if:
       properties:
         compatible:
@@ -118,7 +119,7 @@ allOf:
       properties:
         power-domains: false
 
-additionalProperties: false
+unevaluatedProperties: false
 
 examples:
   - |

-- 
2.55.0

[PATCH RFC 12/12] PoC: arm64: dts: imx93-11x11-frdm: add multiple vdevs

[Francesco Valla]

Add rings for multiple vdevs, as well as the required virtio nodes for
I2C, SPI and GPIO functionalities. On top of that, add example
peripherals using all of them.

NOTE: this is a Proof-Of-Concept, not meant to be integrated!

Signed-off-by: Francesco Valla <francesco@valla.it>
---
 arch/arm64/boot/dts/freescale/imx93-11x11-frdm.dts | 128 +++++++++++++++++++--
 1 file changed, 119 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/boot/dts/freescale/imx93-11x11-frdm.dts b/arch/arm64/boot/dts/freescale/imx93-11x11-frdm.dts
index bd14ba28690c..dfa3b122ac5f 100644
--- a/arch/arm64/boot/dts/freescale/imx93-11x11-frdm.dts
+++ b/arch/arm64/boot/dts/freescale/imx93-11x11-frdm.dts
@@ -53,6 +53,32 @@ button-k3 {
 		};
 	};
 
+	gpio-keys-virtio {
+		compatible = "gpio-keys-polled";
+		poll-interval = <100>;
+
+		button-v1 {
+			label = "Button V1";
+			linux,code = <BTN_3>;
+			gpios = <&v_gpio 23 GPIO_ACTIVE_LOW>;
+		};
+
+		button-v2 {
+			label = "Button V2";
+			linux,code = <BTN_4>;
+			gpios = <&v_gpio 24 GPIO_ACTIVE_LOW>;
+		};
+	};
+
+	leds {
+		compatible = "gpio-leds";
+
+		led {
+			gpios = <&v_gpio 18 GPIO_ACTIVE_HIGH>;
+			label = "LED V";
+		};
+	};
+
 	reg_usdhc2_vmmc: regulator-usdhc2 {
 		compatible = "regulator-fixed";
 		off-on-delay-us = <12000>;
@@ -89,11 +115,6 @@ linux,cma {
 			linux,cma-default;
 		};
 
-		rsc_table: rsc-table@2021e000 {
-			reg = <0 0x2021e000 0 0x1000>;
-			no-map;
-		};
-
 		vdev0vring0: vdev0vring0@a4000000 {
 			reg = <0 0xa4000000 0 0x8000>;
 			no-map;
@@ -105,12 +126,42 @@ vdev0vring1: vdev0vring1@a4008000 {
 		};
 
 		vdev1vring0: vdev1vring0@a4010000 {
-			reg = <0 0xa4010000 0 0x8000>;
+			reg = <0 0xa4010000 0 0x1000>;
+			no-map;
+		};
+
+		vdev2vring0: vdev2vring0@a4011000 {
+			reg = <0 0xa4011000 0 0x2000>;
+			no-map;
+		};
+
+		vdev2vring1: vdev2vring1@a4013000 {
+			reg = <0 0xa4013000 0 0x2000>;
+			no-map;
+		};
+
+		vdev3vring0: vdev3vring0@a4015000 {
+			reg = <0 0xa4015000 0 0x2000>;
+			no-map;
+		};
+
+		vdev4vring0: vdev4vring0@a4017000 {
+			reg = <0 0xa4017000 0 0x4000>;
+			no-map;
+		};
+
+		vdev5vring0: vdev5vring0@a401B000 {
+			reg = <0 0xa401B000 0 0x2000>;
+			no-map;
+		};
+
+		vdev5vring1: vdev5vring1@a401D000 {
+			reg = <0 0xa401D000 0 0x2000>;
 			no-map;
 		};
 
-		vdev1vring1: vdev1vring1@a4018000 {
-			reg = <0 0xa4018000 0 0x8000>;
+		vdev5vring2: vdev5vring2@a401F000 {
+			reg = <0 0xa401F000 0 0x1000>;
 			no-map;
 		};
 
@@ -149,8 +200,67 @@ &cm33 {
 		 <&mu1 3 1>;
 	mbox-names = "tx", "rx", "rxdb";
 	memory-region = <&vdevbuffer>, <&vdev0vring0>, <&vdev0vring1>,
-			<&vdev1vring0>, <&vdev1vring1>, <&rsc_table>;
+			<&vdev1vring0>, <&vdev2vring0>, <&vdev2vring1>,
+			<&vdev3vring0>, <&vdev4vring0>,
+			<&vdev5vring0>, <&vdev5vring1>, <&vdev5vring2>;
 	status = "okay";
+
+	virtio {
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		vdev@2 {
+			reg = <2>;
+
+			v_gpio: gpio {
+				compatible = "virtio,device29";
+				gpio-controller;
+				#gpio-cells = <2>;
+				interrupt-controller;
+				#interrupt-cells = <2>;
+			};
+		};
+
+		vdev@3 {
+			reg = <3>;
+
+			i2c {
+				compatible = "virtio,device22";
+				#address-cells = <1>;
+				#size-cells = <0>;
+
+				eeprom@50 {
+					compatible = "atmel,24c1025";
+					reg = <0x50>;
+				};
+			};
+		};
+
+		vdev@4 {
+			reg = <4>;
+
+			spi {
+				compatible = "virtio,device2d";
+				#address-cells = <1>;
+				#size-cells = <0>;
+
+				sram@0 {
+					compatible = "microchip,mchp23k256";
+					reg = <0>;
+					spi-max-frequency = <20000000>;
+				};
+
+				lcd@1 {
+					compatible = "adafruit,yx240qv29", "ilitek,ili9341";
+					reg = <1>;
+					spi-max-frequency = <10000000>;
+					dc-gpios = <&v_gpio 21 GPIO_ACTIVE_HIGH>;
+					reset-gpios = <&v_gpio 20 GPIO_ACTIVE_HIGH>;
+					rotation = <90>;
+				};
+			};
+		};
+	};
 };
 
 &eqos {

-- 
2.55.0

Re: [PATCH RFC 00/12] remoteproc: add support for any virtio device

[Mathieu Poirier]

Hi Francesco,

I have started reviewing this set but given the size and implications, it will
take me a few days to get through.

Thanks,
Mathieu


On Wed, Sep 16, 2026 at 11:10:45PM +0200, Francesco Valla wrote:
> Hello,
> 
> this patch series introduces the possibility to support generic virtio
> devices over the remotepoc transport, whereas today only rpmsg and
> virtio-console are supported.
> 
> == Introduction ==
> 
> Support for devices other than rpmsg was originally planned [1] and is
> declared inside the documentation [2], but is in practice not there for
> the majority of (if not all) the platforms that provide remoteproc
> capabilities due to memory allocation.
> 
> While vrings are pre-allocated in an area that is reachable by both the
> local (i.e.; Linux) and remote processors, buffers produced by virtio
> drivers aren't, since they typically get allocated through kmalloc.
> The aforementioned rpmsg and virtio-console drivers instead use a trick
> to overcome this limitation and allocate these buffers directly from
> the remoteproc device's coherent memory area, somewhat breaking the
> separation between the driver and the underlying transport.
> 
> == Well, nice, but why? ==
> 
> Main usecase is sharing/virtualization of devices in hypervisor-less
> mixed-criticality contexts, where a subset of peripherals are controlled
> by a "safety" real-time processor but still need to be used by the Linux
> world. Several solutions have been / are being proposed [3] [4], but
> none of them re-uses the existing, standardized virtio specifications. 
> 
> == The proposal ==
> 
> The proposed approach is to introduce a bounce buffering mechanism that
> is transparent to the drivers and can expose to remoteproc devices only
> memory areas they can access. This is obtained by defining the .map
> memeber of each registered vdev and use the map() and unmap() callback
> to bounce data to and from the remote processor, just like the swiotlb
> framework is doing in other contexts, using the device's coherent memory
> area and the associated functions to allocate the bounce buffers.
> 
> During the map() callback the address of the incoming buffer is compared
> against the coherent memory address base and size, to pass through
> buffers already suitable for remoteproc usage (e.g.: the ones allocated
> by the rpmsg framework).
> 
> == Status and open points ==
> 
> The series was tested against a custom Zephyr application [5] running on
> the Cortex-M33 processor of an i.MX93 and exposing six different virtio
> devices:
> 
>   - rpmsg
>   - entropy (rng)
>   - gpio
>   - i2c
>   - spi
>   - can
> 
> On top of three of them (unsurprisingly: i2c, spi, and gpio) several
> devices where declared inside Linux devicetree and successfully used
> (well, technically I'm still experiencing difficulties with gpio
> interrupts not firing on the M33, but that's not really related to the
> series). 
> 
> Several open points are still present, and needs to be either
> investigated or discussed:
> 
>   - for each bounce buffer an entire page is allocated from the coherent
>     memory pool; this is a waste for most of the allocations, which take
>     on average 32 to 64 bytes. An option can be to initialize a DMA pool
>     on one page and allocate small buffers from it?
> 
>   - the support in its current form allocates more memory than before
>     (for bounce buffer tracking) also for existing usecases (i.e.,
>     mainly rpmsg).
> 
>   - an additional issue still exist - and is not solved by this series -
>     for a subset of virtio devices: communication through the device's
>     config space. The remoteproc transport expects this config space to
>     be somewhat constant, and there is no provision to sync changes made
>     by the driver with the remote device. This prevents e.g.
>     virtio-input to work.  
> 
>   - device de-registration on remoteproc stop is causing oopses (under
>     investigation - might no be strictly tied to the series)
> 
> == Patches breakdown ==
> 
> Patches 1 and 2 are cleanups to the remoteproc-virtio driver and could
> be applied independently of this series.
> 
> Patch 3 was submitted a couple of months ago [6] and paves the road for
> the actual support of generic virtio devices, removing the fixed number
> of 2 for the vrings associated to a vdev.
> 
> Patch 4 introduces two new APIs for coherent memory areas associated to
> devices that are used later.
> 
> Patch 5 might somewhat be controversial, as it unconditionally defines
> the VIRTIO_F_VERSION_1 feature for all vdevs. This is required to
> support some virtio device types, and there is no other mean of
> defining it, since the field reserved for features inside the resource
> table is limited to 32 bits. Given that the 1.x virtio specifications
> are ~10 years old this still seems reasonable.
> 
> Patch 6 is were the bounce buffering mechanism is introduced; another
> feature (VIRTIO_F_ACCESS_PLATFORM) is there unconditionally defined to
> force the virtio framework to use the new map APIs.
> 
> Patches 7 and 8 are new devicetree bindings, the first for spi-virtio
> (modelled against the existing ones for gpio-virtio and i2c-virtio) and
> the second for declaring virtio device inside a devicetree. This is not
> required for some devices (e.g.: can, net, gpu), but for others is
> necessary to declare child devices and link them.
> 
> Patch 9 is used to convince the remoteproc-virtio transport to parse the
> bindings just defined; it is worth noting that the virtio framework
> already has the support for devicetree declarations and this adds only
> the glue between the existing support and remoteproc.
> 
> Patches 10 and 11 are i.MX-specific and enable the usage of the newly
> introduced support on this family of platforms. The first one might
> probably be sumbitted as-is independently of the series, as it aligns
> the behavior of imx-rproc to the other platforms in relation to mailbox
> usage.
> 
> Finally, patch 12 is the PoC that has been used to develop and test the
> series and shall not be merged.
> 
> ======
> 
> Thank you in advance for any comment you may want to leave.
> 
> Regards,
> Francesco
> 
> [1] https://lore.kernel.org/all/1330589497-4139-1-git-send-email-ohad@wizery.com/
> [2] https://elixir.bootlin.com/linux/v7.2.5/source/Documentation/staging/remoteproc.rst#L26
> [3] https://lore.kernel.org/linux-remoteproc/20260721204704.400781-1-shenwei.wang@oss.nxp.com/
> [4] https://cfp.embedded-recipes.org/er2026/talk/PCYPJP/
> [5] https://github.com/WallaceIT/zephyr/tree/multi_vdev
> [6] https://lore.kernel.org/all/20260621-vring_flex-v1-1-c6c582fbe94b@valla.it/
> 
> Signed-off-by: Francesco Valla <francesco@valla.it>
> ---
> Francesco Valla (12):
>       remoteproc: virtio: cleanup rproc_add_virtio_dev error path
>       remoteproc: virtio: replace commas with semicolons
>       remoteproc: virtio: support dynamic number of vrings
>       dma-coherent: add base and size APIs
>       remoteproc: always report VIRTIO_F_VERSION_1 feature
>       remoteproc: virtio: add bounce buffering for data buffers
>       dt-bindings: spi: add bindings for spi-virtio
>       dt-bindings: remoteproc: add remoteproc-virtio
>       remoteproc: search for a fwnode during vdev registration
>       remoteproc: imx_rproc: always use non-blocking mailboxes
>       dt-bindings: remoteproc: imx-rproc: support virtio
>       PoC: arm64: dts: imx93-11x11-frdm: add multiple vdevs
> 
>  .../bindings/remoteproc/fsl,imx-rproc.yaml         |   3 +-
>  .../bindings/remoteproc/remoteproc-virtio.yaml     |  89 +++++++++
>  .../devicetree/bindings/spi/spi-virtio.yaml        |  52 +++++
>  arch/arm64/boot/dts/freescale/imx93-11x11-frdm.dts | 128 +++++++++++-
>  drivers/remoteproc/imx_rproc.c                     |  49 +----
>  drivers/remoteproc/imx_rproc.h                     |   1 -
>  drivers/remoteproc/remoteproc_core.c               |  43 +++-
>  drivers/remoteproc/remoteproc_virtio.c             | 222 ++++++++++++++++++---
>  include/linux/dma-map-ops.h                        |  10 +
>  include/linux/remoteproc.h                         |  24 ++-
>  kernel/dma/coherent.c                              |  34 ++++
>  11 files changed, 562 insertions(+), 93 deletions(-)
> ---
> base-commit: 9b87fdc9af2fbfcdb5c24a64139685ef80f6573f
> change-id: 20260915-remoteproc_virtio_map-bcf32a5fab54
> 
> Best regards,
> --  
> Francesco Valla <francesco@valla.it>
> 

Re: [PATCH RFC 00/12] remoteproc: add support for any virtio device

[Francesco Valla]

Hi Mathieu,

On Fri, Sep 18, 2026 at 10:53:54AM -0600, Mathieu Poirier wrote:
> Hi Francesco,
> 
> I have started reviewing this set but given the size and implications, it will
> take me a few days to get through.

I'm going through the review done by sashiko - would you prefer me
sending a v2 with the trivial errors already ironed out before doing
a complete review?

> 
> Thanks,
> Mathieu
> 

Thank you!

Regards,
Francesco