* [PATCH v3 01/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 11:15 ` Laurent Pinchart
2025-04-21 22:26 ` Rob Herring (Arm)
2025-04-17 6:55 ` [PATCH v3 02/13] media: ti: j721e-csi2rx: separate out device and context Rishikesh Donadkar
` (11 subsequent siblings)
12 siblings, 2 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Jai Luthra <j-luthra@ti.com>
The CSI2RX SHIM IP can support 32x DMA channels. These can be used to
split incoming "streams" of data on the CSI-RX port, distinguished by
MIPI Virtual Channel (or Data Type), into different locations in memory.
Actual number of DMA channels allocated to CSI-RX is dependent on the
usecase, and can be modified using the K3 Resource Partitioning tool [1].
So set the minimum channels as 1 and maximum as 32.
Link: https://software-dl.ti.com/processor-sdk-linux/esd/AM62X/10_00_07_04/exports/docs/linux/How_to_Guides/Host/K3_Resource_Partitioning_Tool.html [1]
Link: https://www.ti.com/lit/pdf/spruiv7
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../bindings/media/ti,j721e-csi2rx-shim.yaml | 39 +++++++++++++++++--
1 file changed, 36 insertions(+), 3 deletions(-)
diff --git a/Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml b/Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml
index b9f033f2f3ce4..bf62998b0445a 100644
--- a/Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml
+++ b/Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml
@@ -20,11 +20,44 @@ properties:
const: ti,j721e-csi2rx-shim
dmas:
- maxItems: 1
+ minItems: 1
+ maxItems: 32
dma-names:
+ minItems: 1
items:
- const: rx0
+ - const: rx1
+ - const: rx2
+ - const: rx3
+ - const: rx4
+ - const: rx5
+ - const: rx6
+ - const: rx7
+ - const: rx8
+ - const: rx9
+ - const: rx10
+ - const: rx11
+ - const: rx12
+ - const: rx13
+ - const: rx14
+ - const: rx15
+ - const: rx16
+ - const: rx17
+ - const: rx18
+ - const: rx19
+ - const: rx20
+ - const: rx21
+ - const: rx22
+ - const: rx23
+ - const: rx24
+ - const: rx25
+ - const: rx26
+ - const: rx27
+ - const: rx28
+ - const: rx29
+ - const: rx30
+ - const: rx31
reg:
maxItems: 1
@@ -62,8 +95,8 @@ examples:
ti_csi2rx0: ticsi2rx@4500000 {
compatible = "ti,j721e-csi2rx-shim";
- dmas = <&main_udmap 0x4940>;
- dma-names = "rx0";
+ dmas = <&main_udmap 0x4940>, <&main_udmap 0x4941>;
+ dma-names = "rx0", "rx1";
reg = <0x4500000 0x1000>;
power-domains = <&k3_pds 26 TI_SCI_PD_EXCLUSIVE>;
#address-cells = <1>;
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 01/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans
2025-04-17 6:55 ` [PATCH v3 01/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans Rishikesh Donadkar
@ 2025-04-21 11:15 ` Laurent Pinchart
2025-04-21 22:26 ` Rob Herring (Arm)
1 sibling, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 11:15 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:42PM +0530, Rishikesh Donadkar wrote:
> From: Jai Luthra <j-luthra@ti.com>
>
> The CSI2RX SHIM IP can support 32x DMA channels. These can be used to
> split incoming "streams" of data on the CSI-RX port, distinguished by
> MIPI Virtual Channel (or Data Type), into different locations in memory.
>
> Actual number of DMA channels allocated to CSI-RX is dependent on the
> usecase, and can be modified using the K3 Resource Partitioning tool [1].
> So set the minimum channels as 1 and maximum as 32.
>
> Link: https://software-dl.ti.com/processor-sdk-linux/esd/AM62X/10_00_07_04/exports/docs/linux/How_to_Guides/Host/K3_Resource_Partitioning_Tool.html [1]
> Link: https://www.ti.com/lit/pdf/spruiv7
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> .../bindings/media/ti,j721e-csi2rx-shim.yaml | 39 +++++++++++++++++--
> 1 file changed, 36 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml b/Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml
> index b9f033f2f3ce4..bf62998b0445a 100644
> --- a/Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml
> +++ b/Documentation/devicetree/bindings/media/ti,j721e-csi2rx-shim.yaml
> @@ -20,11 +20,44 @@ properties:
> const: ti,j721e-csi2rx-shim
>
> dmas:
> - maxItems: 1
> + minItems: 1
> + maxItems: 32
>
> dma-names:
> + minItems: 1
> items:
> - const: rx0
> + - const: rx1
> + - const: rx2
> + - const: rx3
> + - const: rx4
> + - const: rx5
> + - const: rx6
> + - const: rx7
> + - const: rx8
> + - const: rx9
> + - const: rx10
> + - const: rx11
> + - const: rx12
> + - const: rx13
> + - const: rx14
> + - const: rx15
> + - const: rx16
> + - const: rx17
> + - const: rx18
> + - const: rx19
> + - const: rx20
> + - const: rx21
> + - const: rx22
> + - const: rx23
> + - const: rx24
> + - const: rx25
> + - const: rx26
> + - const: rx27
> + - const: rx28
> + - const: rx29
> + - const: rx30
> + - const: rx31
>
> reg:
> maxItems: 1
> @@ -62,8 +95,8 @@ examples:
>
> ti_csi2rx0: ticsi2rx@4500000 {
> compatible = "ti,j721e-csi2rx-shim";
> - dmas = <&main_udmap 0x4940>;
> - dma-names = "rx0";
> + dmas = <&main_udmap 0x4940>, <&main_udmap 0x4941>;
> + dma-names = "rx0", "rx1";
> reg = <0x4500000 0x1000>;
> power-domains = <&k3_pds 26 TI_SCI_PD_EXCLUSIVE>;
> #address-cells = <1>;
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 01/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans
2025-04-17 6:55 ` [PATCH v3 01/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans Rishikesh Donadkar
2025-04-21 11:15 ` Laurent Pinchart
@ 2025-04-21 22:26 ` Rob Herring (Arm)
1 sibling, 0 replies; 36+ messages in thread
From: Rob Herring (Arm) @ 2025-04-21 22:26 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: krzk+dt, mchehab, devarsht, hverkuil-cisco, s-jain1, jack.zhu,
vigneshr, linux-media, jai.luthra, laurent.pinchart,
changhuang.liang, linux-kernel, tomi.valkeinen, vaishnav.a,
y-abhilashchandra, jai.luthra, devicetree, mripard, sakari.ailus,
conor+dt
On Thu, 17 Apr 2025 12:25:42 +0530, Rishikesh Donadkar wrote:
> From: Jai Luthra <j-luthra@ti.com>
>
> The CSI2RX SHIM IP can support 32x DMA channels. These can be used to
> split incoming "streams" of data on the CSI-RX port, distinguished by
> MIPI Virtual Channel (or Data Type), into different locations in memory.
>
> Actual number of DMA channels allocated to CSI-RX is dependent on the
> usecase, and can be modified using the K3 Resource Partitioning tool [1].
> So set the minimum channels as 1 and maximum as 32.
>
> Link: https://software-dl.ti.com/processor-sdk-linux/esd/AM62X/10_00_07_04/exports/docs/linux/How_to_Guides/Host/K3_Resource_Partitioning_Tool.html [1]
> Link: https://www.ti.com/lit/pdf/spruiv7
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> .../bindings/media/ti,j721e-csi2rx-shim.yaml | 39 +++++++++++++++++--
> 1 file changed, 36 insertions(+), 3 deletions(-)
>
Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 02/13] media: ti: j721e-csi2rx: separate out device and context
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
2025-04-17 6:55 ` [PATCH v3 01/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 11:32 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 03/13] media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts Rishikesh Donadkar
` (10 subsequent siblings)
12 siblings, 1 reply; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Jai Luthra <j-luthra@ti.com>
The TI CSI2RX wrapper has two parts: the main device and the DMA
contexts. The driver was originally written with single camera capture
in mind, so only one DMA context was needed. For the sake of simplicity,
the context specific stuff was not modeled different to the main device.
To enable multiplexed stream capture, the contexts need to be separated
out from the main device. Create a struct ti_csi2rx_ctx that holds the
DMA context specific things. Separate out functions handling the device
and context related functionality.
Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 412 ++++++++++--------
1 file changed, 228 insertions(+), 184 deletions(-)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index 6412a00be8eab..36cde2e87aabb 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -40,6 +40,8 @@
#define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
#define PSIL_WORD_SIZE_BYTES 16
+#define TI_CSI2RX_NUM_CTX 1
+
/*
* There are no hard limits on the width or height. The DMA engine can handle
* all sizes. The max width and height are arbitrary numbers for this driver.
@@ -64,7 +66,7 @@ struct ti_csi2rx_buffer {
/* Common v4l2 buffer. Must be first. */
struct vb2_v4l2_buffer vb;
struct list_head list;
- struct ti_csi2rx_dev *csi;
+ struct ti_csi2rx_ctx *ctx;
};
enum ti_csi2rx_dma_state {
@@ -84,29 +86,37 @@ struct ti_csi2rx_dma {
* Queue of buffers submitted to DMA engine.
*/
struct list_head submitted;
- /* Buffer to drain stale data from PSI-L endpoint */
- struct {
- void *vaddr;
- dma_addr_t paddr;
- size_t len;
- } drain;
+};
+
+struct ti_csi2rx_dev;
+
+struct ti_csi2rx_ctx {
+ struct ti_csi2rx_dev *csi;
+ struct video_device vdev;
+ struct vb2_queue vidq;
+ struct mutex mutex; /* To serialize ioctls. */
+ struct v4l2_format v_fmt;
+ struct ti_csi2rx_dma dma;
+ u32 sequence;
+ u32 idx;
};
struct ti_csi2rx_dev {
struct device *dev;
void __iomem *shim;
struct v4l2_device v4l2_dev;
- struct video_device vdev;
struct media_device mdev;
struct media_pipeline pipe;
struct media_pad pad;
struct v4l2_async_notifier notifier;
struct v4l2_subdev *source;
- struct vb2_queue vidq;
- struct mutex mutex; /* To serialize ioctls. */
- struct v4l2_format v_fmt;
- struct ti_csi2rx_dma dma;
- u32 sequence;
+ struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
+ /* Buffer to drain stale data from PSI-L endpoint */
+ struct {
+ void *vaddr;
+ dma_addr_t paddr;
+ size_t len;
+ } drain;
};
static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
@@ -212,7 +222,7 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
};
/* Forward declaration needed by ti_csi2rx_dma_callback. */
-static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
+static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
struct ti_csi2rx_buffer *buf);
static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
@@ -302,7 +312,7 @@ static int ti_csi2rx_enum_fmt_vid_cap(struct file *file, void *priv,
static int ti_csi2rx_g_fmt_vid_cap(struct file *file, void *prov,
struct v4l2_format *f)
{
- struct ti_csi2rx_dev *csi = video_drvdata(file);
+ struct ti_csi2rx_ctx *csi = video_drvdata(file);
*f = csi->v_fmt;
@@ -333,7 +343,7 @@ static int ti_csi2rx_try_fmt_vid_cap(struct file *file, void *priv,
static int ti_csi2rx_s_fmt_vid_cap(struct file *file, void *priv,
struct v4l2_format *f)
{
- struct ti_csi2rx_dev *csi = video_drvdata(file);
+ struct ti_csi2rx_ctx *csi = video_drvdata(file);
struct vb2_queue *q = &csi->vidq;
int ret;
@@ -419,25 +429,33 @@ static int csi_async_notifier_bound(struct v4l2_async_notifier *notifier,
static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
{
struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
- struct video_device *vdev = &csi->vdev;
- int ret;
+ int ret, i;
- ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
- if (ret)
- return ret;
-
- ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
- MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
+ for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+ struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
+ struct video_device *vdev = &ctx->vdev;
- if (ret) {
- video_unregister_device(vdev);
- return ret;
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ goto unregister_dev;
}
+ ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret)
+ goto unregister_dev;
+
ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
if (ret)
- video_unregister_device(vdev);
+ goto unregister_dev;
+ return 0;
+
+unregister_dev:
+ i--;
+ for (; i >= 0; i--)
+ video_unregister_device(&csi->ctx[i].vdev);
return ret;
}
@@ -483,12 +501,13 @@ static int ti_csi2rx_notifier_register(struct ti_csi2rx_dev *csi)
return 0;
}
-static void ti_csi2rx_setup_shim(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
{
+ struct ti_csi2rx_dev *csi = ctx->csi;
const struct ti_csi2rx_fmt *fmt;
unsigned int reg;
- fmt = find_format_by_fourcc(csi->v_fmt.fmt.pix.pixelformat);
+ fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
/* De-assert the pixel interface reset. */
reg = SHIM_CNTL_PIX_RST;
@@ -555,8 +574,9 @@ static void ti_csi2rx_drain_callback(void *param)
* To prevent that stale data corrupting the subsequent transactions, it is
* required to issue DMA requests to drain it out.
*/
-static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
+static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
{
+ struct ti_csi2rx_dev *csi = ctx->csi;
struct dma_async_tx_descriptor *desc;
struct completion drain_complete;
dma_cookie_t cookie;
@@ -564,8 +584,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
init_completion(&drain_complete);
- desc = dmaengine_prep_slave_single(csi->dma.chan, csi->dma.drain.paddr,
- csi->dma.drain.len, DMA_DEV_TO_MEM,
+ desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
+ csi->drain.len, DMA_DEV_TO_MEM,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
if (!desc) {
ret = -EIO;
@@ -580,11 +600,11 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
if (ret)
goto out;
- dma_async_issue_pending(csi->dma.chan);
+ dma_async_issue_pending(ctx->dma.chan);
if (!wait_for_completion_timeout(&drain_complete,
msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
- dmaengine_terminate_sync(csi->dma.chan);
+ dmaengine_terminate_sync(ctx->dma.chan);
dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
ret = -ETIMEDOUT;
goto out;
@@ -596,8 +616,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
static void ti_csi2rx_dma_callback(void *param)
{
struct ti_csi2rx_buffer *buf = param;
- struct ti_csi2rx_dev *csi = buf->csi;
- struct ti_csi2rx_dma *dma = &csi->dma;
+ struct ti_csi2rx_ctx *ctx = buf->ctx;
+ struct ti_csi2rx_dma *dma = &ctx->dma;
unsigned long flags;
/*
@@ -605,7 +625,7 @@ static void ti_csi2rx_dma_callback(void *param)
* hardware monitor registers.
*/
buf->vb.vb2_buf.timestamp = ktime_get_ns();
- buf->vb.sequence = csi->sequence++;
+ buf->vb.sequence = ctx->sequence++;
spin_lock_irqsave(&dma->lock, flags);
@@ -617,8 +637,9 @@ static void ti_csi2rx_dma_callback(void *param)
while (!list_empty(&dma->queue)) {
buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
- if (ti_csi2rx_start_dma(csi, buf)) {
- dev_err(csi->dev, "Failed to queue the next buffer for DMA\n");
+ if (ti_csi2rx_start_dma(ctx, buf)) {
+ dev_err(ctx->csi->dev,
+ "Failed to queue the next buffer for DMA\n");
vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
} else {
list_move_tail(&buf->list, &dma->submitted);
@@ -631,17 +652,17 @@ static void ti_csi2rx_dma_callback(void *param)
spin_unlock_irqrestore(&dma->lock, flags);
}
-static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
+static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
struct ti_csi2rx_buffer *buf)
{
unsigned long addr;
struct dma_async_tx_descriptor *desc;
- size_t len = csi->v_fmt.fmt.pix.sizeimage;
+ size_t len = ctx->v_fmt.fmt.pix.sizeimage;
dma_cookie_t cookie;
int ret = 0;
addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
- desc = dmaengine_prep_slave_single(csi->dma.chan, addr, len,
+ desc = dmaengine_prep_slave_single(ctx->dma.chan, addr, len,
DMA_DEV_TO_MEM,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
if (!desc)
@@ -655,20 +676,20 @@ static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
if (ret)
return ret;
- dma_async_issue_pending(csi->dma.chan);
+ dma_async_issue_pending(ctx->dma.chan);
return 0;
}
-static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
{
- struct ti_csi2rx_dma *dma = &csi->dma;
+ struct ti_csi2rx_dma *dma = &ctx->dma;
enum ti_csi2rx_dma_state state;
unsigned long flags;
int ret;
spin_lock_irqsave(&dma->lock, flags);
- state = csi->dma.state;
+ state = ctx->dma.state;
dma->state = TI_CSI2RX_DMA_STOPPED;
spin_unlock_irqrestore(&dma->lock, flags);
@@ -679,30 +700,30 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
* is stopped, as the module-level pixel reset cannot be
* enforced before terminating DMA.
*/
- ret = ti_csi2rx_drain_dma(csi);
+ ret = ti_csi2rx_drain_dma(ctx);
if (ret && ret != -ETIMEDOUT)
- dev_warn(csi->dev,
+ dev_warn(ctx->csi->dev,
"Failed to drain DMA. Next frame might be bogus\n");
}
- ret = dmaengine_terminate_sync(csi->dma.chan);
+ ret = dmaengine_terminate_sync(ctx->dma.chan);
if (ret)
- dev_err(csi->dev, "Failed to stop DMA: %d\n", ret);
+ dev_err(ctx->csi->dev, "Failed to stop DMA: %d\n", ret);
}
-static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_dev *csi,
+static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_ctx *ctx,
enum vb2_buffer_state state)
{
- struct ti_csi2rx_dma *dma = &csi->dma;
+ struct ti_csi2rx_dma *dma = &ctx->dma;
struct ti_csi2rx_buffer *buf, *tmp;
unsigned long flags;
spin_lock_irqsave(&dma->lock, flags);
- list_for_each_entry_safe(buf, tmp, &csi->dma.queue, list) {
+ list_for_each_entry_safe(buf, tmp, &ctx->dma.queue, list) {
list_del(&buf->list);
vb2_buffer_done(&buf->vb.vb2_buf, state);
}
- list_for_each_entry_safe(buf, tmp, &csi->dma.submitted, list) {
+ list_for_each_entry_safe(buf, tmp, &ctx->dma.submitted, list) {
list_del(&buf->list);
vb2_buffer_done(&buf->vb.vb2_buf, state);
}
@@ -713,8 +734,8 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
unsigned int *nplanes, unsigned int sizes[],
struct device *alloc_devs[])
{
- struct ti_csi2rx_dev *csi = vb2_get_drv_priv(q);
- unsigned int size = csi->v_fmt.fmt.pix.sizeimage;
+ struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(q);
+ unsigned int size = ctx->v_fmt.fmt.pix.sizeimage;
if (*nplanes) {
if (sizes[0] < size)
@@ -730,11 +751,11 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
{
- struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
- unsigned long size = csi->v_fmt.fmt.pix.sizeimage;
+ struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ unsigned long size = ctx->v_fmt.fmt.pix.sizeimage;
if (vb2_plane_size(vb, 0) < size) {
- dev_err(csi->dev, "Data will not fit into plane\n");
+ dev_err(ctx->csi->dev, "Data will not fit into plane\n");
return -EINVAL;
}
@@ -744,15 +765,15 @@ static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
{
- struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
+ struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
struct ti_csi2rx_buffer *buf;
- struct ti_csi2rx_dma *dma = &csi->dma;
+ struct ti_csi2rx_dma *dma = &ctx->dma;
bool restart_dma = false;
unsigned long flags = 0;
int ret;
buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
- buf->csi = csi;
+ buf->ctx = ctx;
spin_lock_irqsave(&dma->lock, flags);
/*
@@ -781,18 +802,18 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
* the application and will only confuse it. Issue a DMA
* transaction to drain that up.
*/
- ret = ti_csi2rx_drain_dma(csi);
+ ret = ti_csi2rx_drain_dma(ctx);
if (ret && ret != -ETIMEDOUT)
- dev_warn(csi->dev,
+ dev_warn(ctx->csi->dev,
"Failed to drain DMA. Next frame might be bogus\n");
spin_lock_irqsave(&dma->lock, flags);
- ret = ti_csi2rx_start_dma(csi, buf);
+ ret = ti_csi2rx_start_dma(ctx, buf);
if (ret) {
vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
dma->state = TI_CSI2RX_DMA_IDLE;
spin_unlock_irqrestore(&dma->lock, flags);
- dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
+ dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
} else {
list_add_tail(&buf->list, &dma->submitted);
spin_unlock_irqrestore(&dma->lock, flags);
@@ -802,8 +823,9 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
{
- struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
- struct ti_csi2rx_dma *dma = &csi->dma;
+ struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
+ struct ti_csi2rx_dev *csi = ctx->csi;
+ struct ti_csi2rx_dma *dma = &ctx->dma;
struct ti_csi2rx_buffer *buf;
unsigned long flags;
int ret = 0;
@@ -815,18 +837,18 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
if (ret)
return ret;
- ret = video_device_pipeline_start(&csi->vdev, &csi->pipe);
+ ret = video_device_pipeline_start(&ctx->vdev, &csi->pipe);
if (ret)
goto err;
- ti_csi2rx_setup_shim(csi);
+ ti_csi2rx_setup_shim(ctx);
- csi->sequence = 0;
+ ctx->sequence = 0;
spin_lock_irqsave(&dma->lock, flags);
buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
- ret = ti_csi2rx_start_dma(csi, buf);
+ ret = ti_csi2rx_start_dma(ctx, buf);
if (ret) {
dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
spin_unlock_irqrestore(&dma->lock, flags);
@@ -844,22 +866,23 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
return 0;
err_dma:
- ti_csi2rx_stop_dma(csi);
+ ti_csi2rx_stop_dma(ctx);
err_pipeline:
- video_device_pipeline_stop(&csi->vdev);
+ video_device_pipeline_stop(&ctx->vdev);
writel(0, csi->shim + SHIM_CNTL);
writel(0, csi->shim + SHIM_DMACNTX);
err:
- ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_QUEUED);
+ ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
return ret;
}
static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
{
- struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
+ struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
+ struct ti_csi2rx_dev *csi = ctx->csi;
int ret;
- video_device_pipeline_stop(&csi->vdev);
+ video_device_pipeline_stop(&ctx->vdev);
writel(0, csi->shim + SHIM_CNTL);
writel(0, csi->shim + SHIM_DMACNTX);
@@ -868,8 +891,8 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
if (ret)
dev_err(csi->dev, "Failed to stop subdev stream\n");
- ti_csi2rx_stop_dma(csi);
- ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_ERROR);
+ ti_csi2rx_stop_dma(ctx);
+ ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_ERROR);
}
static const struct vb2_ops csi_vb2_qops = {
@@ -880,27 +903,50 @@ static const struct vb2_ops csi_vb2_qops = {
.stop_streaming = ti_csi2rx_stop_streaming,
};
-static int ti_csi2rx_init_vb2q(struct ti_csi2rx_dev *csi)
+static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
{
- struct vb2_queue *q = &csi->vidq;
+ media_device_unregister(&csi->mdev);
+ v4l2_device_unregister(&csi->v4l2_dev);
+ media_device_cleanup(&csi->mdev);
+}
+
+static void ti_csi2rx_cleanup_notifier(struct ti_csi2rx_dev *csi)
+{
+ v4l2_async_nf_unregister(&csi->notifier);
+ v4l2_async_nf_cleanup(&csi->notifier);
+}
+
+static void ti_csi2rx_cleanup_ctx(struct ti_csi2rx_ctx *ctx)
+{
+ dma_release_channel(ctx->dma.chan);
+ vb2_queue_release(&ctx->vidq);
+
+ video_unregister_device(&ctx->vdev);
+
+ mutex_destroy(&ctx->mutex);
+}
+
+static int ti_csi2rx_init_vb2q(struct ti_csi2rx_ctx *ctx)
+{
+ struct vb2_queue *q = &ctx->vidq;
int ret;
q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
q->io_modes = VB2_MMAP | VB2_DMABUF;
- q->drv_priv = csi;
+ q->drv_priv = ctx;
q->buf_struct_size = sizeof(struct ti_csi2rx_buffer);
q->ops = &csi_vb2_qops;
q->mem_ops = &vb2_dma_contig_memops;
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
- q->dev = dmaengine_get_dma_device(csi->dma.chan);
- q->lock = &csi->mutex;
+ q->dev = dmaengine_get_dma_device(ctx->dma.chan);
+ q->lock = &ctx->mutex;
q->min_queued_buffers = 1;
ret = vb2_queue_init(q);
if (ret)
return ret;
- csi->vdev.queue = q;
+ ctx->vdev.queue = q;
return 0;
}
@@ -909,8 +955,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
{
struct media_entity *entity = link->sink->entity;
struct video_device *vdev = media_entity_to_video_device(entity);
- struct ti_csi2rx_dev *csi = container_of(vdev, struct ti_csi2rx_dev, vdev);
- struct v4l2_pix_format *csi_fmt = &csi->v_fmt.fmt.pix;
+ struct ti_csi2rx_ctx *ctx = container_of(vdev, struct ti_csi2rx_ctx, vdev);
+ struct ti_csi2rx_dev *csi = ctx->csi;
+ struct v4l2_pix_format *csi_fmt = &ctx->v_fmt.fmt.pix;
struct v4l2_subdev_format source_fmt = {
.which = V4L2_SUBDEV_FORMAT_ACTIVE,
.pad = link->source->index,
@@ -963,47 +1010,69 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
.link_validate = ti_csi2rx_link_validate,
};
-static int ti_csi2rx_init_dma(struct ti_csi2rx_dev *csi)
+static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
{
struct dma_slave_config cfg = {
.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
};
int ret;
- INIT_LIST_HEAD(&csi->dma.queue);
- INIT_LIST_HEAD(&csi->dma.submitted);
- spin_lock_init(&csi->dma.lock);
+ INIT_LIST_HEAD(&ctx->dma.queue);
+ INIT_LIST_HEAD(&ctx->dma.submitted);
+ spin_lock_init(&ctx->dma.lock);
- csi->dma.state = TI_CSI2RX_DMA_STOPPED;
+ ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
- csi->dma.chan = dma_request_chan(csi->dev, "rx0");
- if (IS_ERR(csi->dma.chan))
- return PTR_ERR(csi->dma.chan);
+ ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
+ if (IS_ERR(ctx->dma.chan))
+ return PTR_ERR(ctx->dma.chan);
- ret = dmaengine_slave_config(csi->dma.chan, &cfg);
+ ret = dmaengine_slave_config(ctx->dma.chan, &cfg);
if (ret) {
- dma_release_channel(csi->dma.chan);
+ dma_release_channel(ctx->dma.chan);
return ret;
}
- csi->dma.drain.len = DRAIN_BUFFER_SIZE;
- csi->dma.drain.vaddr = dma_alloc_coherent(csi->dev, csi->dma.drain.len,
- &csi->dma.drain.paddr,
- GFP_KERNEL);
- if (!csi->dma.drain.vaddr)
- return -ENOMEM;
-
return 0;
}
static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
{
struct media_device *mdev = &csi->mdev;
- struct video_device *vdev = &csi->vdev;
+ int ret;
+
+ mdev->dev = csi->dev;
+ mdev->hw_revision = 1;
+ strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
+
+ media_device_init(mdev);
+
+ csi->v4l2_dev.mdev = mdev;
+
+ ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
+ if (ret)
+ return ret;
+
+ ret = media_device_register(mdev);
+ if (ret) {
+ v4l2_device_unregister(&csi->v4l2_dev);
+ media_device_cleanup(mdev);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
+{
+ struct ti_csi2rx_dev *csi = ctx->csi;
+ struct video_device *vdev = &ctx->vdev;
const struct ti_csi2rx_fmt *fmt;
- struct v4l2_pix_format *pix_fmt = &csi->v_fmt.fmt.pix;
+ struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
int ret;
+ mutex_init(&ctx->mutex);
+
fmt = find_format_by_fourcc(V4L2_PIX_FMT_UYVY);
if (!fmt)
return -EINVAL;
@@ -1012,19 +1081,20 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
pix_fmt->height = 480;
pix_fmt->field = V4L2_FIELD_NONE;
pix_fmt->colorspace = V4L2_COLORSPACE_SRGB;
- pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
- pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE;
- pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB;
+ pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601,
+ pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE,
+ pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB,
- ti_csi2rx_fill_fmt(fmt, &csi->v_fmt);
+ ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
- mdev->dev = csi->dev;
- mdev->hw_revision = 1;
- strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
-
- media_device_init(mdev);
+ csi->pad.flags = MEDIA_PAD_FL_SINK;
+ vdev->entity.ops = &ti_csi2rx_video_entity_ops;
+ ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+ if (ret)
+ return ret;
- strscpy(vdev->name, TI_CSI2RX_MODULE_NAME, sizeof(vdev->name));
+ snprintf(vdev->name, sizeof(vdev->name), "%s context %u",
+ dev_name(csi->dev), ctx->idx);
vdev->v4l2_dev = &csi->v4l2_dev;
vdev->vfl_dir = VFL_DIR_RX;
vdev->fops = &csi_fops;
@@ -1032,61 +1102,28 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
vdev->release = video_device_release_empty;
vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
V4L2_CAP_IO_MC;
- vdev->lock = &csi->mutex;
- video_set_drvdata(vdev, csi);
+ vdev->lock = &ctx->mutex;
+ video_set_drvdata(vdev, ctx);
- csi->pad.flags = MEDIA_PAD_FL_SINK;
- vdev->entity.ops = &ti_csi2rx_video_entity_ops;
- ret = media_entity_pads_init(&csi->vdev.entity, 1, &csi->pad);
+ ret = ti_csi2rx_init_dma(ctx);
if (ret)
return ret;
- csi->v4l2_dev.mdev = mdev;
-
- ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
+ ret = ti_csi2rx_init_vb2q(ctx);
if (ret)
- return ret;
-
- ret = media_device_register(mdev);
- if (ret) {
- v4l2_device_unregister(&csi->v4l2_dev);
- media_device_cleanup(mdev);
- return ret;
- }
+ goto cleanup_dma;
return 0;
-}
-
-static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_dev *csi)
-{
- dma_free_coherent(csi->dev, csi->dma.drain.len,
- csi->dma.drain.vaddr, csi->dma.drain.paddr);
- csi->dma.drain.vaddr = NULL;
- dma_release_channel(csi->dma.chan);
-}
-
-static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
-{
- media_device_unregister(&csi->mdev);
- v4l2_device_unregister(&csi->v4l2_dev);
- media_device_cleanup(&csi->mdev);
-}
-static void ti_csi2rx_cleanup_subdev(struct ti_csi2rx_dev *csi)
-{
- v4l2_async_nf_unregister(&csi->notifier);
- v4l2_async_nf_cleanup(&csi->notifier);
-}
-
-static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_dev *csi)
-{
- vb2_queue_release(&csi->vidq);
+cleanup_dma:
+ dma_release_channel(ctx->dma.chan);
+ return ret;
}
static int ti_csi2rx_probe(struct platform_device *pdev)
{
struct ti_csi2rx_dev *csi;
- int ret;
+ int ret, i;
csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
if (!csi)
@@ -1095,62 +1132,69 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
csi->dev = &pdev->dev;
platform_set_drvdata(pdev, csi);
- mutex_init(&csi->mutex);
csi->shim = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(csi->shim)) {
ret = PTR_ERR(csi->shim);
- goto err_mutex;
+ return ret;
}
- ret = ti_csi2rx_init_dma(csi);
- if (ret)
- goto err_mutex;
+ csi->drain.len = DRAIN_BUFFER_SIZE;
+ csi->drain.vaddr = dma_alloc_coherent(csi->dev, csi->drain.len,
+ &csi->drain.paddr,
+ GFP_KERNEL);
+ if (!csi->drain.vaddr)
+ return -ENOMEM;
ret = ti_csi2rx_v4l2_init(csi);
- if (ret)
- goto err_dma;
-
- ret = ti_csi2rx_init_vb2q(csi);
if (ret)
goto err_v4l2;
+ for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+ csi->ctx[i].idx = i;
+ csi->ctx[i].csi = csi;
+ ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
+ if (ret)
+ goto err_ctx;
+ }
+
ret = ti_csi2rx_notifier_register(csi);
if (ret)
- goto err_vb2q;
+ goto err_ctx;
ret = of_platform_populate(csi->dev->of_node, NULL, NULL, csi->dev);
if (ret) {
dev_err(csi->dev, "Failed to create children: %d\n", ret);
- goto err_subdev;
+ goto err_notifier;
}
return 0;
-err_subdev:
- ti_csi2rx_cleanup_subdev(csi);
-err_vb2q:
- ti_csi2rx_cleanup_vb2q(csi);
-err_v4l2:
+err_notifier:
+ ti_csi2rx_cleanup_notifier(csi);
+err_ctx:
+ i--;
+ for (; i >= 0; i--)
+ ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
ti_csi2rx_cleanup_v4l2(csi);
-err_dma:
- ti_csi2rx_cleanup_dma(csi);
-err_mutex:
- mutex_destroy(&csi->mutex);
+err_v4l2:
+ dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
+ csi->drain.paddr);
return ret;
}
static void ti_csi2rx_remove(struct platform_device *pdev)
{
struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
+ unsigned int i;
- video_unregister_device(&csi->vdev);
+ for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
+ ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
- ti_csi2rx_cleanup_vb2q(csi);
- ti_csi2rx_cleanup_subdev(csi);
+ ti_csi2rx_cleanup_notifier(csi);
ti_csi2rx_cleanup_v4l2(csi);
- ti_csi2rx_cleanup_dma(csi);
- mutex_destroy(&csi->mutex);
+ dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
+ csi->drain.paddr);
}
static const struct of_device_id ti_csi2rx_of_match[] = {
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 02/13] media: ti: j721e-csi2rx: separate out device and context
2025-04-17 6:55 ` [PATCH v3 02/13] media: ti: j721e-csi2rx: separate out device and context Rishikesh Donadkar
@ 2025-04-21 11:32 ` Laurent Pinchart
2025-04-30 11:01 ` Rishikesh Donadkar
2025-05-09 8:15 ` Rishikesh Donadkar
0 siblings, 2 replies; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 11:32 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
On Thu, Apr 17, 2025 at 12:25:43PM +0530, Rishikesh Donadkar wrote:
> From: Jai Luthra <j-luthra@ti.com>
>
> The TI CSI2RX wrapper has two parts: the main device and the DMA
> contexts. The driver was originally written with single camera capture
> in mind, so only one DMA context was needed. For the sake of simplicity,
> the context specific stuff was not modeled different to the main device.
>
> To enable multiplexed stream capture, the contexts need to be separated
> out from the main device. Create a struct ti_csi2rx_ctx that holds the
> DMA context specific things. Separate out functions handling the device
> and context related functionality.
>
> Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 412 ++++++++++--------
> 1 file changed, 228 insertions(+), 184 deletions(-)
>
> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> index 6412a00be8eab..36cde2e87aabb 100644
> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> @@ -40,6 +40,8 @@
> #define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
>
> #define PSIL_WORD_SIZE_BYTES 16
> +#define TI_CSI2RX_NUM_CTX 1
> +
> /*
> * There are no hard limits on the width or height. The DMA engine can handle
> * all sizes. The max width and height are arbitrary numbers for this driver.
> @@ -64,7 +66,7 @@ struct ti_csi2rx_buffer {
> /* Common v4l2 buffer. Must be first. */
> struct vb2_v4l2_buffer vb;
> struct list_head list;
> - struct ti_csi2rx_dev *csi;
> + struct ti_csi2rx_ctx *ctx;
> };
>
> enum ti_csi2rx_dma_state {
> @@ -84,29 +86,37 @@ struct ti_csi2rx_dma {
> * Queue of buffers submitted to DMA engine.
> */
> struct list_head submitted;
> - /* Buffer to drain stale data from PSI-L endpoint */
> - struct {
> - void *vaddr;
> - dma_addr_t paddr;
> - size_t len;
> - } drain;
> +};
> +
> +struct ti_csi2rx_dev;
> +
> +struct ti_csi2rx_ctx {
> + struct ti_csi2rx_dev *csi;
> + struct video_device vdev;
> + struct vb2_queue vidq;
> + struct mutex mutex; /* To serialize ioctls. */
> + struct v4l2_format v_fmt;
> + struct ti_csi2rx_dma dma;
> + u32 sequence;
> + u32 idx;
> };
>
> struct ti_csi2rx_dev {
> struct device *dev;
> void __iomem *shim;
> struct v4l2_device v4l2_dev;
> - struct video_device vdev;
> struct media_device mdev;
> struct media_pipeline pipe;
> struct media_pad pad;
You need one pad per context, as this models the sink pad of the
video_device.
> struct v4l2_async_notifier notifier;
> struct v4l2_subdev *source;
> - struct vb2_queue vidq;
> - struct mutex mutex; /* To serialize ioctls. */
> - struct v4l2_format v_fmt;
> - struct ti_csi2rx_dma dma;
> - u32 sequence;
> + struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
> + /* Buffer to drain stale data from PSI-L endpoint */
> + struct {
> + void *vaddr;
> + dma_addr_t paddr;
> + size_t len;
> + } drain;
> };
>
> static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
> @@ -212,7 +222,7 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
> };
>
> /* Forward declaration needed by ti_csi2rx_dma_callback. */
> -static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
> +static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
> struct ti_csi2rx_buffer *buf);
>
> static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
> @@ -302,7 +312,7 @@ static int ti_csi2rx_enum_fmt_vid_cap(struct file *file, void *priv,
> static int ti_csi2rx_g_fmt_vid_cap(struct file *file, void *prov,
> struct v4l2_format *f)
> {
> - struct ti_csi2rx_dev *csi = video_drvdata(file);
> + struct ti_csi2rx_ctx *csi = video_drvdata(file);
>
> *f = csi->v_fmt;
>
> @@ -333,7 +343,7 @@ static int ti_csi2rx_try_fmt_vid_cap(struct file *file, void *priv,
> static int ti_csi2rx_s_fmt_vid_cap(struct file *file, void *priv,
> struct v4l2_format *f)
> {
> - struct ti_csi2rx_dev *csi = video_drvdata(file);
> + struct ti_csi2rx_ctx *csi = video_drvdata(file);
> struct vb2_queue *q = &csi->vidq;
> int ret;
>
> @@ -419,25 +429,33 @@ static int csi_async_notifier_bound(struct v4l2_async_notifier *notifier,
> static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
> {
> struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
> - struct video_device *vdev = &csi->vdev;
> - int ret;
> + int ret, i;
>
> - ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> - if (ret)
> - return ret;
> -
> - ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
> - MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
> + struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
> + struct video_device *vdev = &ctx->vdev;
>
> - if (ret) {
> - video_unregister_device(vdev);
> - return ret;
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret)
> + goto unregister_dev;
> }
>
> + ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
> + MEDIA_LNK_FL_IMMUTABLE |
> + MEDIA_LNK_FL_ENABLED);
This will become problematic... It gets fixed in patch 05/13. Could you
reorder patches to add the subdev first ?
> + if (ret)
> + goto unregister_dev;
> +
> ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
> if (ret)
> - video_unregister_device(vdev);
> + goto unregister_dev;
>
> + return 0;
> +
> +unregister_dev:
> + i--;
> + for (; i >= 0; i--)
> + video_unregister_device(&csi->ctx[i].vdev);
> return ret;
> }
>
> @@ -483,12 +501,13 @@ static int ti_csi2rx_notifier_register(struct ti_csi2rx_dev *csi)
> return 0;
> }
>
> -static void ti_csi2rx_setup_shim(struct ti_csi2rx_dev *csi)
> +static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
> {
> + struct ti_csi2rx_dev *csi = ctx->csi;
> const struct ti_csi2rx_fmt *fmt;
> unsigned int reg;
>
> - fmt = find_format_by_fourcc(csi->v_fmt.fmt.pix.pixelformat);
> + fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
>
> /* De-assert the pixel interface reset. */
> reg = SHIM_CNTL_PIX_RST;
> @@ -555,8 +574,9 @@ static void ti_csi2rx_drain_callback(void *param)
> * To prevent that stale data corrupting the subsequent transactions, it is
> * required to issue DMA requests to drain it out.
> */
> -static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
> +static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
> {
> + struct ti_csi2rx_dev *csi = ctx->csi;
> struct dma_async_tx_descriptor *desc;
> struct completion drain_complete;
> dma_cookie_t cookie;
> @@ -564,8 +584,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>
> init_completion(&drain_complete);
>
> - desc = dmaengine_prep_slave_single(csi->dma.chan, csi->dma.drain.paddr,
> - csi->dma.drain.len, DMA_DEV_TO_MEM,
> + desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
> + csi->drain.len, DMA_DEV_TO_MEM,
> DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
> if (!desc) {
> ret = -EIO;
> @@ -580,11 +600,11 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
> if (ret)
> goto out;
>
> - dma_async_issue_pending(csi->dma.chan);
> + dma_async_issue_pending(ctx->dma.chan);
>
> if (!wait_for_completion_timeout(&drain_complete,
> msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
> - dmaengine_terminate_sync(csi->dma.chan);
> + dmaengine_terminate_sync(ctx->dma.chan);
> dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
> ret = -ETIMEDOUT;
> goto out;
> @@ -596,8 +616,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
> static void ti_csi2rx_dma_callback(void *param)
> {
> struct ti_csi2rx_buffer *buf = param;
> - struct ti_csi2rx_dev *csi = buf->csi;
> - struct ti_csi2rx_dma *dma = &csi->dma;
> + struct ti_csi2rx_ctx *ctx = buf->ctx;
> + struct ti_csi2rx_dma *dma = &ctx->dma;
> unsigned long flags;
>
> /*
> @@ -605,7 +625,7 @@ static void ti_csi2rx_dma_callback(void *param)
> * hardware monitor registers.
> */
> buf->vb.vb2_buf.timestamp = ktime_get_ns();
> - buf->vb.sequence = csi->sequence++;
> + buf->vb.sequence = ctx->sequence++;
>
> spin_lock_irqsave(&dma->lock, flags);
>
> @@ -617,8 +637,9 @@ static void ti_csi2rx_dma_callback(void *param)
> while (!list_empty(&dma->queue)) {
> buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
>
> - if (ti_csi2rx_start_dma(csi, buf)) {
> - dev_err(csi->dev, "Failed to queue the next buffer for DMA\n");
> + if (ti_csi2rx_start_dma(ctx, buf)) {
> + dev_err(ctx->csi->dev,
> + "Failed to queue the next buffer for DMA\n");
Printing some sort of context identifier would be useful here. Same
below where applicable.
> vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> } else {
> list_move_tail(&buf->list, &dma->submitted);
> @@ -631,17 +652,17 @@ static void ti_csi2rx_dma_callback(void *param)
> spin_unlock_irqrestore(&dma->lock, flags);
> }
>
> -static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
> +static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
> struct ti_csi2rx_buffer *buf)
> {
> unsigned long addr;
> struct dma_async_tx_descriptor *desc;
> - size_t len = csi->v_fmt.fmt.pix.sizeimage;
> + size_t len = ctx->v_fmt.fmt.pix.sizeimage;
> dma_cookie_t cookie;
> int ret = 0;
>
> addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> - desc = dmaengine_prep_slave_single(csi->dma.chan, addr, len,
> + desc = dmaengine_prep_slave_single(ctx->dma.chan, addr, len,
> DMA_DEV_TO_MEM,
> DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
> if (!desc)
> @@ -655,20 +676,20 @@ static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
> if (ret)
> return ret;
>
> - dma_async_issue_pending(csi->dma.chan);
> + dma_async_issue_pending(ctx->dma.chan);
>
> return 0;
> }
>
> -static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
> +static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
> {
> - struct ti_csi2rx_dma *dma = &csi->dma;
> + struct ti_csi2rx_dma *dma = &ctx->dma;
> enum ti_csi2rx_dma_state state;
> unsigned long flags;
> int ret;
>
> spin_lock_irqsave(&dma->lock, flags);
> - state = csi->dma.state;
> + state = ctx->dma.state;
> dma->state = TI_CSI2RX_DMA_STOPPED;
> spin_unlock_irqrestore(&dma->lock, flags);
>
> @@ -679,30 +700,30 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
> * is stopped, as the module-level pixel reset cannot be
> * enforced before terminating DMA.
> */
> - ret = ti_csi2rx_drain_dma(csi);
> + ret = ti_csi2rx_drain_dma(ctx);
> if (ret && ret != -ETIMEDOUT)
> - dev_warn(csi->dev,
> + dev_warn(ctx->csi->dev,
> "Failed to drain DMA. Next frame might be bogus\n");
> }
>
> - ret = dmaengine_terminate_sync(csi->dma.chan);
> + ret = dmaengine_terminate_sync(ctx->dma.chan);
> if (ret)
> - dev_err(csi->dev, "Failed to stop DMA: %d\n", ret);
> + dev_err(ctx->csi->dev, "Failed to stop DMA: %d\n", ret);
> }
>
> -static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_dev *csi,
> +static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_ctx *ctx,
> enum vb2_buffer_state state)
> {
> - struct ti_csi2rx_dma *dma = &csi->dma;
> + struct ti_csi2rx_dma *dma = &ctx->dma;
> struct ti_csi2rx_buffer *buf, *tmp;
> unsigned long flags;
>
> spin_lock_irqsave(&dma->lock, flags);
> - list_for_each_entry_safe(buf, tmp, &csi->dma.queue, list) {
> + list_for_each_entry_safe(buf, tmp, &ctx->dma.queue, list) {
> list_del(&buf->list);
> vb2_buffer_done(&buf->vb.vb2_buf, state);
> }
> - list_for_each_entry_safe(buf, tmp, &csi->dma.submitted, list) {
> + list_for_each_entry_safe(buf, tmp, &ctx->dma.submitted, list) {
> list_del(&buf->list);
> vb2_buffer_done(&buf->vb.vb2_buf, state);
> }
> @@ -713,8 +734,8 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
> unsigned int *nplanes, unsigned int sizes[],
> struct device *alloc_devs[])
> {
> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(q);
> - unsigned int size = csi->v_fmt.fmt.pix.sizeimage;
> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(q);
> + unsigned int size = ctx->v_fmt.fmt.pix.sizeimage;
>
> if (*nplanes) {
> if (sizes[0] < size)
> @@ -730,11 +751,11 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
>
> static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
> {
> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
> - unsigned long size = csi->v_fmt.fmt.pix.sizeimage;
> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> + unsigned long size = ctx->v_fmt.fmt.pix.sizeimage;
>
> if (vb2_plane_size(vb, 0) < size) {
> - dev_err(csi->dev, "Data will not fit into plane\n");
> + dev_err(ctx->csi->dev, "Data will not fit into plane\n");
> return -EINVAL;
> }
>
> @@ -744,15 +765,15 @@ static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
>
> static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
> {
> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> struct ti_csi2rx_buffer *buf;
> - struct ti_csi2rx_dma *dma = &csi->dma;
> + struct ti_csi2rx_dma *dma = &ctx->dma;
> bool restart_dma = false;
> unsigned long flags = 0;
> int ret;
>
> buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
> - buf->csi = csi;
> + buf->ctx = ctx;
>
> spin_lock_irqsave(&dma->lock, flags);
> /*
> @@ -781,18 +802,18 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
> * the application and will only confuse it. Issue a DMA
> * transaction to drain that up.
> */
> - ret = ti_csi2rx_drain_dma(csi);
> + ret = ti_csi2rx_drain_dma(ctx);
> if (ret && ret != -ETIMEDOUT)
> - dev_warn(csi->dev,
> + dev_warn(ctx->csi->dev,
> "Failed to drain DMA. Next frame might be bogus\n");
>
> spin_lock_irqsave(&dma->lock, flags);
> - ret = ti_csi2rx_start_dma(csi, buf);
> + ret = ti_csi2rx_start_dma(ctx, buf);
> if (ret) {
> vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> dma->state = TI_CSI2RX_DMA_IDLE;
> spin_unlock_irqrestore(&dma->lock, flags);
> - dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
> + dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
> } else {
> list_add_tail(&buf->list, &dma->submitted);
> spin_unlock_irqrestore(&dma->lock, flags);
> @@ -802,8 +823,9 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
>
> static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
> {
> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
> - struct ti_csi2rx_dma *dma = &csi->dma;
> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
> + struct ti_csi2rx_dev *csi = ctx->csi;
> + struct ti_csi2rx_dma *dma = &ctx->dma;
> struct ti_csi2rx_buffer *buf;
> unsigned long flags;
> int ret = 0;
> @@ -815,18 +837,18 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
> if (ret)
> return ret;
>
> - ret = video_device_pipeline_start(&csi->vdev, &csi->pipe);
> + ret = video_device_pipeline_start(&ctx->vdev, &csi->pipe);
> if (ret)
> goto err;
>
> - ti_csi2rx_setup_shim(csi);
> + ti_csi2rx_setup_shim(ctx);
>
> - csi->sequence = 0;
> + ctx->sequence = 0;
>
> spin_lock_irqsave(&dma->lock, flags);
> buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
>
> - ret = ti_csi2rx_start_dma(csi, buf);
> + ret = ti_csi2rx_start_dma(ctx, buf);
> if (ret) {
> dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
> spin_unlock_irqrestore(&dma->lock, flags);
> @@ -844,22 +866,23 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
> return 0;
>
> err_dma:
> - ti_csi2rx_stop_dma(csi);
> + ti_csi2rx_stop_dma(ctx);
> err_pipeline:
> - video_device_pipeline_stop(&csi->vdev);
> + video_device_pipeline_stop(&ctx->vdev);
> writel(0, csi->shim + SHIM_CNTL);
> writel(0, csi->shim + SHIM_DMACNTX);
> err:
> - ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_QUEUED);
> + ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
> return ret;
> }
>
> static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
> {
> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
> + struct ti_csi2rx_dev *csi = ctx->csi;
> int ret;
>
> - video_device_pipeline_stop(&csi->vdev);
> + video_device_pipeline_stop(&ctx->vdev);
>
> writel(0, csi->shim + SHIM_CNTL);
> writel(0, csi->shim + SHIM_DMACNTX);
> @@ -868,8 +891,8 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
> if (ret)
> dev_err(csi->dev, "Failed to stop subdev stream\n");
>
> - ti_csi2rx_stop_dma(csi);
> - ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_ERROR);
> + ti_csi2rx_stop_dma(ctx);
> + ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_ERROR);
> }
>
> static const struct vb2_ops csi_vb2_qops = {
> @@ -880,27 +903,50 @@ static const struct vb2_ops csi_vb2_qops = {
> .stop_streaming = ti_csi2rx_stop_streaming,
> };
>
> -static int ti_csi2rx_init_vb2q(struct ti_csi2rx_dev *csi)
> +static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
> {
> - struct vb2_queue *q = &csi->vidq;
> + media_device_unregister(&csi->mdev);
> + v4l2_device_unregister(&csi->v4l2_dev);
> + media_device_cleanup(&csi->mdev);
> +}
> +
> +static void ti_csi2rx_cleanup_notifier(struct ti_csi2rx_dev *csi)
> +{
> + v4l2_async_nf_unregister(&csi->notifier);
> + v4l2_async_nf_cleanup(&csi->notifier);
> +}
> +
> +static void ti_csi2rx_cleanup_ctx(struct ti_csi2rx_ctx *ctx)
> +{
> + dma_release_channel(ctx->dma.chan);
> + vb2_queue_release(&ctx->vidq);
> +
> + video_unregister_device(&ctx->vdev);
> +
> + mutex_destroy(&ctx->mutex);
> +}
> +
> +static int ti_csi2rx_init_vb2q(struct ti_csi2rx_ctx *ctx)
> +{
> + struct vb2_queue *q = &ctx->vidq;
> int ret;
>
> q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> q->io_modes = VB2_MMAP | VB2_DMABUF;
> - q->drv_priv = csi;
> + q->drv_priv = ctx;
> q->buf_struct_size = sizeof(struct ti_csi2rx_buffer);
> q->ops = &csi_vb2_qops;
> q->mem_ops = &vb2_dma_contig_memops;
> q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> - q->dev = dmaengine_get_dma_device(csi->dma.chan);
> - q->lock = &csi->mutex;
> + q->dev = dmaengine_get_dma_device(ctx->dma.chan);
> + q->lock = &ctx->mutex;
> q->min_queued_buffers = 1;
>
> ret = vb2_queue_init(q);
> if (ret)
> return ret;
>
> - csi->vdev.queue = q;
> + ctx->vdev.queue = q;
>
> return 0;
> }
> @@ -909,8 +955,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
> {
> struct media_entity *entity = link->sink->entity;
> struct video_device *vdev = media_entity_to_video_device(entity);
> - struct ti_csi2rx_dev *csi = container_of(vdev, struct ti_csi2rx_dev, vdev);
> - struct v4l2_pix_format *csi_fmt = &csi->v_fmt.fmt.pix;
> + struct ti_csi2rx_ctx *ctx = container_of(vdev, struct ti_csi2rx_ctx, vdev);
> + struct ti_csi2rx_dev *csi = ctx->csi;
> + struct v4l2_pix_format *csi_fmt = &ctx->v_fmt.fmt.pix;
> struct v4l2_subdev_format source_fmt = {
> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> .pad = link->source->index,
> @@ -963,47 +1010,69 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
> .link_validate = ti_csi2rx_link_validate,
> };
>
> -static int ti_csi2rx_init_dma(struct ti_csi2rx_dev *csi)
> +static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
> {
> struct dma_slave_config cfg = {
> .src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
> };
> int ret;
>
> - INIT_LIST_HEAD(&csi->dma.queue);
> - INIT_LIST_HEAD(&csi->dma.submitted);
> - spin_lock_init(&csi->dma.lock);
> + INIT_LIST_HEAD(&ctx->dma.queue);
> + INIT_LIST_HEAD(&ctx->dma.submitted);
> + spin_lock_init(&ctx->dma.lock);
>
> - csi->dma.state = TI_CSI2RX_DMA_STOPPED;
> + ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
>
> - csi->dma.chan = dma_request_chan(csi->dev, "rx0");
> - if (IS_ERR(csi->dma.chan))
> - return PTR_ERR(csi->dma.chan);
> + ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
> + if (IS_ERR(ctx->dma.chan))
> + return PTR_ERR(ctx->dma.chan);
>
> - ret = dmaengine_slave_config(csi->dma.chan, &cfg);
> + ret = dmaengine_slave_config(ctx->dma.chan, &cfg);
> if (ret) {
> - dma_release_channel(csi->dma.chan);
> + dma_release_channel(ctx->dma.chan);
> return ret;
> }
>
> - csi->dma.drain.len = DRAIN_BUFFER_SIZE;
> - csi->dma.drain.vaddr = dma_alloc_coherent(csi->dev, csi->dma.drain.len,
> - &csi->dma.drain.paddr,
> - GFP_KERNEL);
> - if (!csi->dma.drain.vaddr)
> - return -ENOMEM;
> -
> return 0;
> }
>
> static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
> {
> struct media_device *mdev = &csi->mdev;
> - struct video_device *vdev = &csi->vdev;
> + int ret;
> +
> + mdev->dev = csi->dev;
> + mdev->hw_revision = 1;
> + strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
> +
> + media_device_init(mdev);
> +
> + csi->v4l2_dev.mdev = mdev;
> +
> + ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
> + if (ret)
> + return ret;
> +
> + ret = media_device_register(mdev);
> + if (ret) {
> + v4l2_device_unregister(&csi->v4l2_dev);
> + media_device_cleanup(mdev);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
> +{
> + struct ti_csi2rx_dev *csi = ctx->csi;
> + struct video_device *vdev = &ctx->vdev;
> const struct ti_csi2rx_fmt *fmt;
> - struct v4l2_pix_format *pix_fmt = &csi->v_fmt.fmt.pix;
> + struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
> int ret;
>
> + mutex_init(&ctx->mutex);
> +
> fmt = find_format_by_fourcc(V4L2_PIX_FMT_UYVY);
> if (!fmt)
> return -EINVAL;
> @@ -1012,19 +1081,20 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
> pix_fmt->height = 480;
> pix_fmt->field = V4L2_FIELD_NONE;
> pix_fmt->colorspace = V4L2_COLORSPACE_SRGB;
> - pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
> - pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE;
> - pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB;
> + pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601,
> + pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE,
> + pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB,
>
> - ti_csi2rx_fill_fmt(fmt, &csi->v_fmt);
> + ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
>
> - mdev->dev = csi->dev;
> - mdev->hw_revision = 1;
> - strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
> -
> - media_device_init(mdev);
> + csi->pad.flags = MEDIA_PAD_FL_SINK;
> + vdev->entity.ops = &ti_csi2rx_video_entity_ops;
> + ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
> + if (ret)
> + return ret;
>
> - strscpy(vdev->name, TI_CSI2RX_MODULE_NAME, sizeof(vdev->name));
> + snprintf(vdev->name, sizeof(vdev->name), "%s context %u",
> + dev_name(csi->dev), ctx->idx);
> vdev->v4l2_dev = &csi->v4l2_dev;
> vdev->vfl_dir = VFL_DIR_RX;
> vdev->fops = &csi_fops;
> @@ -1032,61 +1102,28 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
> vdev->release = video_device_release_empty;
> vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
> V4L2_CAP_IO_MC;
> - vdev->lock = &csi->mutex;
> - video_set_drvdata(vdev, csi);
> + vdev->lock = &ctx->mutex;
> + video_set_drvdata(vdev, ctx);
>
> - csi->pad.flags = MEDIA_PAD_FL_SINK;
> - vdev->entity.ops = &ti_csi2rx_video_entity_ops;
> - ret = media_entity_pads_init(&csi->vdev.entity, 1, &csi->pad);
> + ret = ti_csi2rx_init_dma(ctx);
> if (ret)
> return ret;
>
> - csi->v4l2_dev.mdev = mdev;
> -
> - ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
> + ret = ti_csi2rx_init_vb2q(ctx);
> if (ret)
> - return ret;
> -
> - ret = media_device_register(mdev);
> - if (ret) {
> - v4l2_device_unregister(&csi->v4l2_dev);
> - media_device_cleanup(mdev);
> - return ret;
> - }
> + goto cleanup_dma;
>
> return 0;
> -}
> -
> -static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_dev *csi)
> -{
> - dma_free_coherent(csi->dev, csi->dma.drain.len,
> - csi->dma.drain.vaddr, csi->dma.drain.paddr);
> - csi->dma.drain.vaddr = NULL;
> - dma_release_channel(csi->dma.chan);
> -}
> -
> -static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
> -{
> - media_device_unregister(&csi->mdev);
> - v4l2_device_unregister(&csi->v4l2_dev);
> - media_device_cleanup(&csi->mdev);
> -}
>
> -static void ti_csi2rx_cleanup_subdev(struct ti_csi2rx_dev *csi)
> -{
> - v4l2_async_nf_unregister(&csi->notifier);
> - v4l2_async_nf_cleanup(&csi->notifier);
> -}
> -
> -static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_dev *csi)
> -{
> - vb2_queue_release(&csi->vidq);
> +cleanup_dma:
> + dma_release_channel(ctx->dma.chan);
> + return ret;
> }
>
> static int ti_csi2rx_probe(struct platform_device *pdev)
> {
> struct ti_csi2rx_dev *csi;
> - int ret;
> + int ret, i;
>
> csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
> if (!csi)
> @@ -1095,62 +1132,69 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
> csi->dev = &pdev->dev;
> platform_set_drvdata(pdev, csi);
>
> - mutex_init(&csi->mutex);
> csi->shim = devm_platform_ioremap_resource(pdev, 0);
> if (IS_ERR(csi->shim)) {
> ret = PTR_ERR(csi->shim);
> - goto err_mutex;
> + return ret;
> }
>
> - ret = ti_csi2rx_init_dma(csi);
> - if (ret)
> - goto err_mutex;
> + csi->drain.len = DRAIN_BUFFER_SIZE;
> + csi->drain.vaddr = dma_alloc_coherent(csi->dev, csi->drain.len,
> + &csi->drain.paddr,
> + GFP_KERNEL);
> + if (!csi->drain.vaddr)
> + return -ENOMEM;
>
> ret = ti_csi2rx_v4l2_init(csi);
> - if (ret)
> - goto err_dma;
> -
> - ret = ti_csi2rx_init_vb2q(csi);
> if (ret)
> goto err_v4l2;
>
> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
> + csi->ctx[i].idx = i;
> + csi->ctx[i].csi = csi;
> + ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
> + if (ret)
> + goto err_ctx;
> + }
> +
> ret = ti_csi2rx_notifier_register(csi);
> if (ret)
> - goto err_vb2q;
> + goto err_ctx;
>
> ret = of_platform_populate(csi->dev->of_node, NULL, NULL, csi->dev);
> if (ret) {
> dev_err(csi->dev, "Failed to create children: %d\n", ret);
> - goto err_subdev;
> + goto err_notifier;
> }
>
> return 0;
>
> -err_subdev:
> - ti_csi2rx_cleanup_subdev(csi);
> -err_vb2q:
> - ti_csi2rx_cleanup_vb2q(csi);
> -err_v4l2:
> +err_notifier:
> + ti_csi2rx_cleanup_notifier(csi);
> +err_ctx:
> + i--;
> + for (; i >= 0; i--)
> + ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
> ti_csi2rx_cleanup_v4l2(csi);
> -err_dma:
> - ti_csi2rx_cleanup_dma(csi);
> -err_mutex:
> - mutex_destroy(&csi->mutex);
> +err_v4l2:
> + dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
> + csi->drain.paddr);
> return ret;
> }
>
> static void ti_csi2rx_remove(struct platform_device *pdev)
> {
> struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
> + unsigned int i;
>
> - video_unregister_device(&csi->vdev);
> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
> + ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
>
> - ti_csi2rx_cleanup_vb2q(csi);
> - ti_csi2rx_cleanup_subdev(csi);
> + ti_csi2rx_cleanup_notifier(csi);
> ti_csi2rx_cleanup_v4l2(csi);
> - ti_csi2rx_cleanup_dma(csi);
>
> - mutex_destroy(&csi->mutex);
> + dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
> + csi->drain.paddr);
> }
>
> static const struct of_device_id ti_csi2rx_of_match[] = {
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 02/13] media: ti: j721e-csi2rx: separate out device and context
2025-04-21 11:32 ` Laurent Pinchart
@ 2025-04-30 11:01 ` Rishikesh Donadkar
2025-05-09 8:15 ` Rishikesh Donadkar
1 sibling, 0 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-30 11:01 UTC (permalink / raw)
To: Laurent Pinchart
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
On 21/04/25 17:02, Laurent Pinchart wrote:
> On Thu, Apr 17, 2025 at 12:25:43PM +0530, Rishikesh Donadkar wrote:
>> From: Jai Luthra <j-luthra@ti.com>
>>
>> The TI CSI2RX wrapper has two parts: the main device and the DMA
>> contexts. The driver was originally written with single camera capture
>> in mind, so only one DMA context was needed. For the sake of simplicity,
>> the context specific stuff was not modeled different to the main device.
>>
>> To enable multiplexed stream capture, the contexts need to be separated
>> out from the main device. Create a struct ti_csi2rx_ctx that holds the
>> DMA context specific things. Separate out functions handling the device
>> and context related functionality.
>>
>> Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
>> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
>> Signed-off-by: Jai Luthra <j-luthra@ti.com>
>> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
>> ---
>> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 412 ++++++++++--------
>> 1 file changed, 228 insertions(+), 184 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> index 6412a00be8eab..36cde2e87aabb 100644
>> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> @@ -40,6 +40,8 @@
>> #define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
>>
>> #define PSIL_WORD_SIZE_BYTES 16
>> +#define TI_CSI2RX_NUM_CTX 1
>> +
>> /*
>> * There are no hard limits on the width or height. The DMA engine can handle
>> * all sizes. The max width and height are arbitrary numbers for this driver.
>> @@ -64,7 +66,7 @@ struct ti_csi2rx_buffer {
>> /* Common v4l2 buffer. Must be first. */
>> struct vb2_v4l2_buffer vb;
>> struct list_head list;
>> - struct ti_csi2rx_dev *csi;
>> + struct ti_csi2rx_ctx *ctx;
>> };
>>
>> enum ti_csi2rx_dma_state {
>> @@ -84,29 +86,37 @@ struct ti_csi2rx_dma {
>> * Queue of buffers submitted to DMA engine.
>> */
>> struct list_head submitted;
>> - /* Buffer to drain stale data from PSI-L endpoint */
>> - struct {
>> - void *vaddr;
>> - dma_addr_t paddr;
>> - size_t len;
>> - } drain;
>> +};
>> +
>> +struct ti_csi2rx_dev;
>> +
>> +struct ti_csi2rx_ctx {
>> + struct ti_csi2rx_dev *csi;
>> + struct video_device vdev;
>> + struct vb2_queue vidq;
>> + struct mutex mutex; /* To serialize ioctls. */
>> + struct v4l2_format v_fmt;
>> + struct ti_csi2rx_dma dma;
>> + u32 sequence;
>> + u32 idx;
>> };
>>
>> struct ti_csi2rx_dev {
>> struct device *dev;
>> void __iomem *shim;
>> struct v4l2_device v4l2_dev;
>> - struct video_device vdev;
>> struct media_device mdev;
>> struct media_pipeline pipe;
>> struct media_pad pad;
> You need one pad per context, as this models the sink pad of the
> video_device.
I will fix it in the next revision
>
>> struct v4l2_async_notifier notifier;
>> struct v4l2_subdev *source;
>> - struct vb2_queue vidq;
>> - struct mutex mutex; /* To serialize ioctls. */
>> - struct v4l2_format v_fmt;
>> - struct ti_csi2rx_dma dma;
>> - u32 sequence;
>> + struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
>> + /* Buffer to drain stale data from PSI-L endpoint */
>> + struct {
>> + void *vaddr;
>> + dma_addr_t paddr;
>> + size_t len;
>> + } drain;
>> };
>>
>> static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
>> @@ -212,7 +222,7 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
>> };
>>
>> /* Forward declaration needed by ti_csi2rx_dma_callback. */
>> -static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
>> +static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
>> struct ti_csi2rx_buffer *buf);
>>
>> static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
>> @@ -302,7 +312,7 @@ static int ti_csi2rx_enum_fmt_vid_cap(struct file *file, void *priv,
>> static int ti_csi2rx_g_fmt_vid_cap(struct file *file, void *prov,
>> struct v4l2_format *f)
>> {
>> - struct ti_csi2rx_dev *csi = video_drvdata(file);
>> + struct ti_csi2rx_ctx *csi = video_drvdata(file);
>>
>> *f = csi->v_fmt;
>>
>> @@ -333,7 +343,7 @@ static int ti_csi2rx_try_fmt_vid_cap(struct file *file, void *priv,
>> static int ti_csi2rx_s_fmt_vid_cap(struct file *file, void *priv,
>> struct v4l2_format *f)
>> {
>> - struct ti_csi2rx_dev *csi = video_drvdata(file);
>> + struct ti_csi2rx_ctx *csi = video_drvdata(file);
>> struct vb2_queue *q = &csi->vidq;
>> int ret;
>>
>> @@ -419,25 +429,33 @@ static int csi_async_notifier_bound(struct v4l2_async_notifier *notifier,
>> static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
>> {
>> struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
>> - struct video_device *vdev = &csi->vdev;
>> - int ret;
>> + int ret, i;
>>
>> - ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> - if (ret)
>> - return ret;
>> -
>> - ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
>> - MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
>> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
>> + struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
>> + struct video_device *vdev = &ctx->vdev;
>>
>> - if (ret) {
>> - video_unregister_device(vdev);
>> - return ret;
>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> + if (ret)
>> + goto unregister_dev;
>> }
>>
>> + ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
>> + MEDIA_LNK_FL_IMMUTABLE |
>> + MEDIA_LNK_FL_ENABLED);
> This will become problematic... It gets fixed in patch 05/13. Could you
> reorder patches to add the subdev first ?
Yes
>
>> + if (ret)
>> + goto unregister_dev;
>> +
>> ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
>> if (ret)
>> - video_unregister_device(vdev);
>> + goto unregister_dev;
>>
>> + return 0;
>> +
>> +unregister_dev:
>> + i--;
>> + for (; i >= 0; i--)
>> + video_unregister_device(&csi->ctx[i].vdev);
>> return ret;
>> }
>>
>> @@ -483,12 +501,13 @@ static int ti_csi2rx_notifier_register(struct ti_csi2rx_dev *csi)
>> return 0;
>> }
>>
>> -static void ti_csi2rx_setup_shim(struct ti_csi2rx_dev *csi)
>> +static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
>> {
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> const struct ti_csi2rx_fmt *fmt;
>> unsigned int reg;
>>
>> - fmt = find_format_by_fourcc(csi->v_fmt.fmt.pix.pixelformat);
>> + fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
>>
>> /* De-assert the pixel interface reset. */
>> reg = SHIM_CNTL_PIX_RST;
>> @@ -555,8 +574,9 @@ static void ti_csi2rx_drain_callback(void *param)
>> * To prevent that stale data corrupting the subsequent transactions, it is
>> * required to issue DMA requests to drain it out.
>> */
>> -static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>> +static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
>> {
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> struct dma_async_tx_descriptor *desc;
>> struct completion drain_complete;
>> dma_cookie_t cookie;
>> @@ -564,8 +584,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>>
>> init_completion(&drain_complete);
>>
>> - desc = dmaengine_prep_slave_single(csi->dma.chan, csi->dma.drain.paddr,
>> - csi->dma.drain.len, DMA_DEV_TO_MEM,
>> + desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
>> + csi->drain.len, DMA_DEV_TO_MEM,
>> DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
>> if (!desc) {
>> ret = -EIO;
>> @@ -580,11 +600,11 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>> if (ret)
>> goto out;
>>
>> - dma_async_issue_pending(csi->dma.chan);
>> + dma_async_issue_pending(ctx->dma.chan);
>>
>> if (!wait_for_completion_timeout(&drain_complete,
>> msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
>> - dmaengine_terminate_sync(csi->dma.chan);
>> + dmaengine_terminate_sync(ctx->dma.chan);
>> dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
>> ret = -ETIMEDOUT;
>> goto out;
>> @@ -596,8 +616,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>> static void ti_csi2rx_dma_callback(void *param)
>> {
>> struct ti_csi2rx_buffer *buf = param;
>> - struct ti_csi2rx_dev *csi = buf->csi;
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_ctx *ctx = buf->ctx;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> unsigned long flags;
>>
>> /*
>> @@ -605,7 +625,7 @@ static void ti_csi2rx_dma_callback(void *param)
>> * hardware monitor registers.
>> */
>> buf->vb.vb2_buf.timestamp = ktime_get_ns();
>> - buf->vb.sequence = csi->sequence++;
>> + buf->vb.sequence = ctx->sequence++;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>>
>> @@ -617,8 +637,9 @@ static void ti_csi2rx_dma_callback(void *param)
>> while (!list_empty(&dma->queue)) {
>> buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
>>
>> - if (ti_csi2rx_start_dma(csi, buf)) {
>> - dev_err(csi->dev, "Failed to queue the next buffer for DMA\n");
>> + if (ti_csi2rx_start_dma(ctx, buf)) {
>> + dev_err(ctx->csi->dev,
>> + "Failed to queue the next buffer for DMA\n");
> Printing some sort of context identifier would be useful here. Same
> below where applicable.
Okay, will add it in next revision
>
>> vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> } else {
>> list_move_tail(&buf->list, &dma->submitted);
>> @@ -631,17 +652,17 @@ static void ti_csi2rx_dma_callback(void *param)
>> spin_unlock_irqrestore(&dma->lock, flags);
>> }
>>
>> -static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
>> +static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
>> struct ti_csi2rx_buffer *buf)
>> {
>> unsigned long addr;
>> struct dma_async_tx_descriptor *desc;
>> - size_t len = csi->v_fmt.fmt.pix.sizeimage;
>> + size_t len = ctx->v_fmt.fmt.pix.sizeimage;
>> dma_cookie_t cookie;
>> int ret = 0;
>>
>> addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
>> - desc = dmaengine_prep_slave_single(csi->dma.chan, addr, len,
>> + desc = dmaengine_prep_slave_single(ctx->dma.chan, addr, len,
>> DMA_DEV_TO_MEM,
>> DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
>> if (!desc)
>> @@ -655,20 +676,20 @@ static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
>> if (ret)
>> return ret;
>>
>> - dma_async_issue_pending(csi->dma.chan);
>> + dma_async_issue_pending(ctx->dma.chan);
>>
>> return 0;
>> }
>>
>> -static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
>> +static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
>> {
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> enum ti_csi2rx_dma_state state;
>> unsigned long flags;
>> int ret;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> - state = csi->dma.state;
>> + state = ctx->dma.state;
>> dma->state = TI_CSI2RX_DMA_STOPPED;
>> spin_unlock_irqrestore(&dma->lock, flags);
>>
>> @@ -679,30 +700,30 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
>> * is stopped, as the module-level pixel reset cannot be
>> * enforced before terminating DMA.
>> */
>> - ret = ti_csi2rx_drain_dma(csi);
>> + ret = ti_csi2rx_drain_dma(ctx);
>> if (ret && ret != -ETIMEDOUT)
>> - dev_warn(csi->dev,
>> + dev_warn(ctx->csi->dev,
>> "Failed to drain DMA. Next frame might be bogus\n");
>> }
>>
>> - ret = dmaengine_terminate_sync(csi->dma.chan);
>> + ret = dmaengine_terminate_sync(ctx->dma.chan);
>> if (ret)
>> - dev_err(csi->dev, "Failed to stop DMA: %d\n", ret);
>> + dev_err(ctx->csi->dev, "Failed to stop DMA: %d\n", ret);
>> }
>>
>> -static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_dev *csi,
>> +static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_ctx *ctx,
>> enum vb2_buffer_state state)
>> {
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> struct ti_csi2rx_buffer *buf, *tmp;
>> unsigned long flags;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> - list_for_each_entry_safe(buf, tmp, &csi->dma.queue, list) {
>> + list_for_each_entry_safe(buf, tmp, &ctx->dma.queue, list) {
>> list_del(&buf->list);
>> vb2_buffer_done(&buf->vb.vb2_buf, state);
>> }
>> - list_for_each_entry_safe(buf, tmp, &csi->dma.submitted, list) {
>> + list_for_each_entry_safe(buf, tmp, &ctx->dma.submitted, list) {
>> list_del(&buf->list);
>> vb2_buffer_done(&buf->vb.vb2_buf, state);
>> }
>> @@ -713,8 +734,8 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
>> unsigned int *nplanes, unsigned int sizes[],
>> struct device *alloc_devs[])
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(q);
>> - unsigned int size = csi->v_fmt.fmt.pix.sizeimage;
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(q);
>> + unsigned int size = ctx->v_fmt.fmt.pix.sizeimage;
>>
>> if (*nplanes) {
>> if (sizes[0] < size)
>> @@ -730,11 +751,11 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
>>
>> static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
>> - unsigned long size = csi->v_fmt.fmt.pix.sizeimage;
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
>> + unsigned long size = ctx->v_fmt.fmt.pix.sizeimage;
>>
>> if (vb2_plane_size(vb, 0) < size) {
>> - dev_err(csi->dev, "Data will not fit into plane\n");
>> + dev_err(ctx->csi->dev, "Data will not fit into plane\n");
>> return -EINVAL;
>> }
>>
>> @@ -744,15 +765,15 @@ static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
>>
>> static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
>> struct ti_csi2rx_buffer *buf;
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> bool restart_dma = false;
>> unsigned long flags = 0;
>> int ret;
>>
>> buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
>> - buf->csi = csi;
>> + buf->ctx = ctx;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> /*
>> @@ -781,18 +802,18 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
>> * the application and will only confuse it. Issue a DMA
>> * transaction to drain that up.
>> */
>> - ret = ti_csi2rx_drain_dma(csi);
>> + ret = ti_csi2rx_drain_dma(ctx);
>> if (ret && ret != -ETIMEDOUT)
>> - dev_warn(csi->dev,
>> + dev_warn(ctx->csi->dev,
>> "Failed to drain DMA. Next frame might be bogus\n");
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> - ret = ti_csi2rx_start_dma(csi, buf);
>> + ret = ti_csi2rx_start_dma(ctx, buf);
>> if (ret) {
>> vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> dma->state = TI_CSI2RX_DMA_IDLE;
>> spin_unlock_irqrestore(&dma->lock, flags);
>> - dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
>> + dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
>> } else {
>> list_add_tail(&buf->list, &dma->submitted);
>> spin_unlock_irqrestore(&dma->lock, flags);
>> @@ -802,8 +823,9 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
>>
>> static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> struct ti_csi2rx_buffer *buf;
>> unsigned long flags;
>> int ret = 0;
>> @@ -815,18 +837,18 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> if (ret)
>> return ret;
>>
>> - ret = video_device_pipeline_start(&csi->vdev, &csi->pipe);
>> + ret = video_device_pipeline_start(&ctx->vdev, &csi->pipe);
>> if (ret)
>> goto err;
>>
>> - ti_csi2rx_setup_shim(csi);
>> + ti_csi2rx_setup_shim(ctx);
>>
>> - csi->sequence = 0;
>> + ctx->sequence = 0;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
>>
>> - ret = ti_csi2rx_start_dma(csi, buf);
>> + ret = ti_csi2rx_start_dma(ctx, buf);
>> if (ret) {
>> dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
>> spin_unlock_irqrestore(&dma->lock, flags);
>> @@ -844,22 +866,23 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> return 0;
>>
>> err_dma:
>> - ti_csi2rx_stop_dma(csi);
>> + ti_csi2rx_stop_dma(ctx);
>> err_pipeline:
>> - video_device_pipeline_stop(&csi->vdev);
>> + video_device_pipeline_stop(&ctx->vdev);
>> writel(0, csi->shim + SHIM_CNTL);
>> writel(0, csi->shim + SHIM_DMACNTX);
>> err:
>> - ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_QUEUED);
>> + ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
>> return ret;
>> }
>>
>> static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> int ret;
>>
>> - video_device_pipeline_stop(&csi->vdev);
>> + video_device_pipeline_stop(&ctx->vdev);
>>
>> writel(0, csi->shim + SHIM_CNTL);
>> writel(0, csi->shim + SHIM_DMACNTX);
>> @@ -868,8 +891,8 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
>> if (ret)
>> dev_err(csi->dev, "Failed to stop subdev stream\n");
>>
>> - ti_csi2rx_stop_dma(csi);
>> - ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_ERROR);
>> + ti_csi2rx_stop_dma(ctx);
>> + ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_ERROR);
>> }
>>
>> static const struct vb2_ops csi_vb2_qops = {
>> @@ -880,27 +903,50 @@ static const struct vb2_ops csi_vb2_qops = {
>> .stop_streaming = ti_csi2rx_stop_streaming,
>> };
>>
>> -static int ti_csi2rx_init_vb2q(struct ti_csi2rx_dev *csi)
>> +static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
>> {
>> - struct vb2_queue *q = &csi->vidq;
>> + media_device_unregister(&csi->mdev);
>> + v4l2_device_unregister(&csi->v4l2_dev);
>> + media_device_cleanup(&csi->mdev);
>> +}
>> +
>> +static void ti_csi2rx_cleanup_notifier(struct ti_csi2rx_dev *csi)
>> +{
>> + v4l2_async_nf_unregister(&csi->notifier);
>> + v4l2_async_nf_cleanup(&csi->notifier);
>> +}
>> +
>> +static void ti_csi2rx_cleanup_ctx(struct ti_csi2rx_ctx *ctx)
>> +{
>> + dma_release_channel(ctx->dma.chan);
>> + vb2_queue_release(&ctx->vidq);
>> +
>> + video_unregister_device(&ctx->vdev);
>> +
>> + mutex_destroy(&ctx->mutex);
>> +}
>> +
>> +static int ti_csi2rx_init_vb2q(struct ti_csi2rx_ctx *ctx)
>> +{
>> + struct vb2_queue *q = &ctx->vidq;
>> int ret;
>>
>> q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>> q->io_modes = VB2_MMAP | VB2_DMABUF;
>> - q->drv_priv = csi;
>> + q->drv_priv = ctx;
>> q->buf_struct_size = sizeof(struct ti_csi2rx_buffer);
>> q->ops = &csi_vb2_qops;
>> q->mem_ops = &vb2_dma_contig_memops;
>> q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> - q->dev = dmaengine_get_dma_device(csi->dma.chan);
>> - q->lock = &csi->mutex;
>> + q->dev = dmaengine_get_dma_device(ctx->dma.chan);
>> + q->lock = &ctx->mutex;
>> q->min_queued_buffers = 1;
>>
>> ret = vb2_queue_init(q);
>> if (ret)
>> return ret;
>>
>> - csi->vdev.queue = q;
>> + ctx->vdev.queue = q;
>>
>> return 0;
>> }
>> @@ -909,8 +955,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
>> {
>> struct media_entity *entity = link->sink->entity;
>> struct video_device *vdev = media_entity_to_video_device(entity);
>> - struct ti_csi2rx_dev *csi = container_of(vdev, struct ti_csi2rx_dev, vdev);
>> - struct v4l2_pix_format *csi_fmt = &csi->v_fmt.fmt.pix;
>> + struct ti_csi2rx_ctx *ctx = container_of(vdev, struct ti_csi2rx_ctx, vdev);
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> + struct v4l2_pix_format *csi_fmt = &ctx->v_fmt.fmt.pix;
>> struct v4l2_subdev_format source_fmt = {
>> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>> .pad = link->source->index,
>> @@ -963,47 +1010,69 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
>> .link_validate = ti_csi2rx_link_validate,
>> };
>>
>> -static int ti_csi2rx_init_dma(struct ti_csi2rx_dev *csi)
>> +static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
>> {
>> struct dma_slave_config cfg = {
>> .src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
>> };
>> int ret;
>>
>> - INIT_LIST_HEAD(&csi->dma.queue);
>> - INIT_LIST_HEAD(&csi->dma.submitted);
>> - spin_lock_init(&csi->dma.lock);
>> + INIT_LIST_HEAD(&ctx->dma.queue);
>> + INIT_LIST_HEAD(&ctx->dma.submitted);
>> + spin_lock_init(&ctx->dma.lock);
>>
>> - csi->dma.state = TI_CSI2RX_DMA_STOPPED;
>> + ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
>>
>> - csi->dma.chan = dma_request_chan(csi->dev, "rx0");
>> - if (IS_ERR(csi->dma.chan))
>> - return PTR_ERR(csi->dma.chan);
>> + ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
>> + if (IS_ERR(ctx->dma.chan))
>> + return PTR_ERR(ctx->dma.chan);
>>
>> - ret = dmaengine_slave_config(csi->dma.chan, &cfg);
>> + ret = dmaengine_slave_config(ctx->dma.chan, &cfg);
>> if (ret) {
>> - dma_release_channel(csi->dma.chan);
>> + dma_release_channel(ctx->dma.chan);
>> return ret;
>> }
>>
>> - csi->dma.drain.len = DRAIN_BUFFER_SIZE;
>> - csi->dma.drain.vaddr = dma_alloc_coherent(csi->dev, csi->dma.drain.len,
>> - &csi->dma.drain.paddr,
>> - GFP_KERNEL);
>> - if (!csi->dma.drain.vaddr)
>> - return -ENOMEM;
>> -
>> return 0;
>> }
>>
>> static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>> {
>> struct media_device *mdev = &csi->mdev;
>> - struct video_device *vdev = &csi->vdev;
>> + int ret;
>> +
>> + mdev->dev = csi->dev;
>> + mdev->hw_revision = 1;
>> + strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
>> +
>> + media_device_init(mdev);
>> +
>> + csi->v4l2_dev.mdev = mdev;
>> +
>> + ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
>> + if (ret)
>> + return ret;
>> +
>> + ret = media_device_register(mdev);
>> + if (ret) {
>> + v4l2_device_unregister(&csi->v4l2_dev);
>> + media_device_cleanup(mdev);
>> + return ret;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
>> +{
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> + struct video_device *vdev = &ctx->vdev;
>> const struct ti_csi2rx_fmt *fmt;
>> - struct v4l2_pix_format *pix_fmt = &csi->v_fmt.fmt.pix;
>> + struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
>> int ret;
>>
>> + mutex_init(&ctx->mutex);
>> +
>> fmt = find_format_by_fourcc(V4L2_PIX_FMT_UYVY);
>> if (!fmt)
>> return -EINVAL;
>> @@ -1012,19 +1081,20 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>> pix_fmt->height = 480;
>> pix_fmt->field = V4L2_FIELD_NONE;
>> pix_fmt->colorspace = V4L2_COLORSPACE_SRGB;
>> - pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
>> - pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE;
>> - pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB;
>> + pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601,
>> + pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE,
>> + pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB,
>>
>> - ti_csi2rx_fill_fmt(fmt, &csi->v_fmt);
>> + ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
>>
>> - mdev->dev = csi->dev;
>> - mdev->hw_revision = 1;
>> - strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
>> -
>> - media_device_init(mdev);
>> + csi->pad.flags = MEDIA_PAD_FL_SINK;
>> + vdev->entity.ops = &ti_csi2rx_video_entity_ops;
>> + ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
>> + if (ret)
>> + return ret;
>>
>> - strscpy(vdev->name, TI_CSI2RX_MODULE_NAME, sizeof(vdev->name));
>> + snprintf(vdev->name, sizeof(vdev->name), "%s context %u",
>> + dev_name(csi->dev), ctx->idx);
>> vdev->v4l2_dev = &csi->v4l2_dev;
>> vdev->vfl_dir = VFL_DIR_RX;
>> vdev->fops = &csi_fops;
>> @@ -1032,61 +1102,28 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>> vdev->release = video_device_release_empty;
>> vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
>> V4L2_CAP_IO_MC;
>> - vdev->lock = &csi->mutex;
>> - video_set_drvdata(vdev, csi);
>> + vdev->lock = &ctx->mutex;
>> + video_set_drvdata(vdev, ctx);
>>
>> - csi->pad.flags = MEDIA_PAD_FL_SINK;
>> - vdev->entity.ops = &ti_csi2rx_video_entity_ops;
>> - ret = media_entity_pads_init(&csi->vdev.entity, 1, &csi->pad);
>> + ret = ti_csi2rx_init_dma(ctx);
>> if (ret)
>> return ret;
>>
>> - csi->v4l2_dev.mdev = mdev;
>> -
>> - ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
>> + ret = ti_csi2rx_init_vb2q(ctx);
>> if (ret)
>> - return ret;
>> -
>> - ret = media_device_register(mdev);
>> - if (ret) {
>> - v4l2_device_unregister(&csi->v4l2_dev);
>> - media_device_cleanup(mdev);
>> - return ret;
>> - }
>> + goto cleanup_dma;
>>
>> return 0;
>> -}
>> -
>> -static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_dev *csi)
>> -{
>> - dma_free_coherent(csi->dev, csi->dma.drain.len,
>> - csi->dma.drain.vaddr, csi->dma.drain.paddr);
>> - csi->dma.drain.vaddr = NULL;
>> - dma_release_channel(csi->dma.chan);
>> -}
>> -
>> -static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
>> -{
>> - media_device_unregister(&csi->mdev);
>> - v4l2_device_unregister(&csi->v4l2_dev);
>> - media_device_cleanup(&csi->mdev);
>> -}
>>
>> -static void ti_csi2rx_cleanup_subdev(struct ti_csi2rx_dev *csi)
>> -{
>> - v4l2_async_nf_unregister(&csi->notifier);
>> - v4l2_async_nf_cleanup(&csi->notifier);
>> -}
>> -
>> -static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_dev *csi)
>> -{
>> - vb2_queue_release(&csi->vidq);
>> +cleanup_dma:
>> + dma_release_channel(ctx->dma.chan);
>> + return ret;
>> }
>>
>> static int ti_csi2rx_probe(struct platform_device *pdev)
>> {
>> struct ti_csi2rx_dev *csi;
>> - int ret;
>> + int ret, i;
>>
>> csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
>> if (!csi)
>> @@ -1095,62 +1132,69 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
>> csi->dev = &pdev->dev;
>> platform_set_drvdata(pdev, csi);
>>
>> - mutex_init(&csi->mutex);
>> csi->shim = devm_platform_ioremap_resource(pdev, 0);
>> if (IS_ERR(csi->shim)) {
>> ret = PTR_ERR(csi->shim);
>> - goto err_mutex;
>> + return ret;
>> }
>>
>> - ret = ti_csi2rx_init_dma(csi);
>> - if (ret)
>> - goto err_mutex;
>> + csi->drain.len = DRAIN_BUFFER_SIZE;
>> + csi->drain.vaddr = dma_alloc_coherent(csi->dev, csi->drain.len,
>> + &csi->drain.paddr,
>> + GFP_KERNEL);
>> + if (!csi->drain.vaddr)
>> + return -ENOMEM;
>>
>> ret = ti_csi2rx_v4l2_init(csi);
>> - if (ret)
>> - goto err_dma;
>> -
>> - ret = ti_csi2rx_init_vb2q(csi);
>> if (ret)
>> goto err_v4l2;
>>
>> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
>> + csi->ctx[i].idx = i;
>> + csi->ctx[i].csi = csi;
>> + ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
>> + if (ret)
>> + goto err_ctx;
>> + }
>> +
>> ret = ti_csi2rx_notifier_register(csi);
>> if (ret)
>> - goto err_vb2q;
>> + goto err_ctx;
>>
>> ret = of_platform_populate(csi->dev->of_node, NULL, NULL, csi->dev);
>> if (ret) {
>> dev_err(csi->dev, "Failed to create children: %d\n", ret);
>> - goto err_subdev;
>> + goto err_notifier;
>> }
>>
>> return 0;
>>
>> -err_subdev:
>> - ti_csi2rx_cleanup_subdev(csi);
>> -err_vb2q:
>> - ti_csi2rx_cleanup_vb2q(csi);
>> -err_v4l2:
>> +err_notifier:
>> + ti_csi2rx_cleanup_notifier(csi);
>> +err_ctx:
>> + i--;
>> + for (; i >= 0; i--)
>> + ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
>> ti_csi2rx_cleanup_v4l2(csi);
>> -err_dma:
>> - ti_csi2rx_cleanup_dma(csi);
>> -err_mutex:
>> - mutex_destroy(&csi->mutex);
>> +err_v4l2:
>> + dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
>> + csi->drain.paddr);
>> return ret;
>> }
>>
>> static void ti_csi2rx_remove(struct platform_device *pdev)
>> {
>> struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
>> + unsigned int i;
>>
>> - video_unregister_device(&csi->vdev);
>> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
>> + ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
>>
>> - ti_csi2rx_cleanup_vb2q(csi);
>> - ti_csi2rx_cleanup_subdev(csi);
>> + ti_csi2rx_cleanup_notifier(csi);
>> ti_csi2rx_cleanup_v4l2(csi);
>> - ti_csi2rx_cleanup_dma(csi);
>>
>> - mutex_destroy(&csi->mutex);
>> + dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
>> + csi->drain.paddr);
>> }
>>
>> static const struct of_device_id ti_csi2rx_of_match[] = {
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 02/13] media: ti: j721e-csi2rx: separate out device and context
2025-04-21 11:32 ` Laurent Pinchart
2025-04-30 11:01 ` Rishikesh Donadkar
@ 2025-05-09 8:15 ` Rishikesh Donadkar
1 sibling, 0 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-05-09 8:15 UTC (permalink / raw)
To: Laurent Pinchart
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
On 21/04/25 17:02, Laurent Pinchart wrote:
> On Thu, Apr 17, 2025 at 12:25:43PM +0530, Rishikesh Donadkar wrote:
>> From: Jai Luthra <j-luthra@ti.com>
>>
>> The TI CSI2RX wrapper has two parts: the main device and the DMA
>> contexts. The driver was originally written with single camera capture
>> in mind, so only one DMA context was needed. For the sake of simplicity,
>> the context specific stuff was not modeled different to the main device.
>>
>> To enable multiplexed stream capture, the contexts need to be separated
>> out from the main device. Create a struct ti_csi2rx_ctx that holds the
>> DMA context specific things. Separate out functions handling the device
>> and context related functionality.
>>
>> Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
>> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
>> Signed-off-by: Jai Luthra <j-luthra@ti.com>
>> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
>> ---
>> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 412 ++++++++++--------
>> 1 file changed, 228 insertions(+), 184 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> index 6412a00be8eab..36cde2e87aabb 100644
>> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> @@ -40,6 +40,8 @@
>> #define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
>>
>> #define PSIL_WORD_SIZE_BYTES 16
>> +#define TI_CSI2RX_NUM_CTX 1
>> +
>> /*
>> * There are no hard limits on the width or height. The DMA engine can handle
>> * all sizes. The max width and height are arbitrary numbers for this driver.
>> @@ -64,7 +66,7 @@ struct ti_csi2rx_buffer {
>> /* Common v4l2 buffer. Must be first. */
>> struct vb2_v4l2_buffer vb;
>> struct list_head list;
>> - struct ti_csi2rx_dev *csi;
>> + struct ti_csi2rx_ctx *ctx;
>> };
>>
>> enum ti_csi2rx_dma_state {
>> @@ -84,29 +86,37 @@ struct ti_csi2rx_dma {
>> * Queue of buffers submitted to DMA engine.
>> */
>> struct list_head submitted;
>> - /* Buffer to drain stale data from PSI-L endpoint */
>> - struct {
>> - void *vaddr;
>> - dma_addr_t paddr;
>> - size_t len;
>> - } drain;
>> +};
>> +
>> +struct ti_csi2rx_dev;
>> +
>> +struct ti_csi2rx_ctx {
>> + struct ti_csi2rx_dev *csi;
>> + struct video_device vdev;
>> + struct vb2_queue vidq;
>> + struct mutex mutex; /* To serialize ioctls. */
>> + struct v4l2_format v_fmt;
>> + struct ti_csi2rx_dma dma;
>> + u32 sequence;
>> + u32 idx;
>> };
>>
>> struct ti_csi2rx_dev {
>> struct device *dev;
>> void __iomem *shim;
>> struct v4l2_device v4l2_dev;
>> - struct video_device vdev;
>> struct media_device mdev;
>> struct media_pipeline pipe;
>> struct media_pad pad;
> You need one pad per context, as this models the sink pad of the
> video_device.
>> struct v4l2_async_notifier notifier;
>> struct v4l2_subdev *source;
>> - struct vb2_queue vidq;
>> - struct mutex mutex; /* To serialize ioctls. */
>> - struct v4l2_format v_fmt;
>> - struct ti_csi2rx_dma dma;
>> - u32 sequence;
>> + struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
>> + /* Buffer to drain stale data from PSI-L endpoint */
>> + struct {
>> + void *vaddr;
>> + dma_addr_t paddr;
>> + size_t len;
>> + } drain;
>> };
>>
>> static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
>> @@ -212,7 +222,7 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
>> };
>>
>> /* Forward declaration needed by ti_csi2rx_dma_callback. */
>> -static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
>> +static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
>> struct ti_csi2rx_buffer *buf);
>>
>> static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
>> @@ -302,7 +312,7 @@ static int ti_csi2rx_enum_fmt_vid_cap(struct file *file, void *priv,
>> static int ti_csi2rx_g_fmt_vid_cap(struct file *file, void *prov,
>> struct v4l2_format *f)
>> {
>> - struct ti_csi2rx_dev *csi = video_drvdata(file);
>> + struct ti_csi2rx_ctx *csi = video_drvdata(file);
>>
>> *f = csi->v_fmt;
>>
>> @@ -333,7 +343,7 @@ static int ti_csi2rx_try_fmt_vid_cap(struct file *file, void *priv,
>> static int ti_csi2rx_s_fmt_vid_cap(struct file *file, void *priv,
>> struct v4l2_format *f)
>> {
>> - struct ti_csi2rx_dev *csi = video_drvdata(file);
>> + struct ti_csi2rx_ctx *csi = video_drvdata(file);
>> struct vb2_queue *q = &csi->vidq;
>> int ret;
>>
>> @@ -419,25 +429,33 @@ static int csi_async_notifier_bound(struct v4l2_async_notifier *notifier,
>> static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
>> {
>> struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
>> - struct video_device *vdev = &csi->vdev;
>> - int ret;
>> + int ret, i;
>>
>> - ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> - if (ret)
>> - return ret;
>> -
>> - ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
>> - MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
>> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
>> + struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
>> + struct video_device *vdev = &ctx->vdev;
>>
>> - if (ret) {
>> - video_unregister_device(vdev);
>> - return ret;
>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> + if (ret)
>> + goto unregister_dev;
>> }
>>
>> + ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
>> + MEDIA_LNK_FL_IMMUTABLE |
>> + MEDIA_LNK_FL_ENABLED);
> This will become problematic... It gets fixed in patch 05/13. Could you
> reorder patches to add the subdev first ?
Hi Laurent,
Adding the subdev first will allow using the media_create_pad_link()
API. But since cdns-csi2rx bridge has multiple source pad, fwnode APIs
still needs to be used so that link with the right source pad of
cdns-csi2rx bridge can be created.
v4l2_create_fwnode_links_to_pad() does that already.
Regards,
Rishikesh
>
>> + if (ret)
>> + goto unregister_dev;
>> +
>> ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
>> if (ret)
>> - video_unregister_device(vdev);
>> + goto unregister_dev;
>>
>> + return 0;
>> +
>> +unregister_dev:
>> + i--;
>> + for (; i >= 0; i--)
>> + video_unregister_device(&csi->ctx[i].vdev);
>> return ret;
>> }
>>
>> @@ -483,12 +501,13 @@ static int ti_csi2rx_notifier_register(struct ti_csi2rx_dev *csi)
>> return 0;
>> }
>>
>> -static void ti_csi2rx_setup_shim(struct ti_csi2rx_dev *csi)
>> +static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
>> {
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> const struct ti_csi2rx_fmt *fmt;
>> unsigned int reg;
>>
>> - fmt = find_format_by_fourcc(csi->v_fmt.fmt.pix.pixelformat);
>> + fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
>>
>> /* De-assert the pixel interface reset. */
>> reg = SHIM_CNTL_PIX_RST;
>> @@ -555,8 +574,9 @@ static void ti_csi2rx_drain_callback(void *param)
>> * To prevent that stale data corrupting the subsequent transactions, it is
>> * required to issue DMA requests to drain it out.
>> */
>> -static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>> +static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
>> {
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> struct dma_async_tx_descriptor *desc;
>> struct completion drain_complete;
>> dma_cookie_t cookie;
>> @@ -564,8 +584,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>>
>> init_completion(&drain_complete);
>>
>> - desc = dmaengine_prep_slave_single(csi->dma.chan, csi->dma.drain.paddr,
>> - csi->dma.drain.len, DMA_DEV_TO_MEM,
>> + desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
>> + csi->drain.len, DMA_DEV_TO_MEM,
>> DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
>> if (!desc) {
>> ret = -EIO;
>> @@ -580,11 +600,11 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>> if (ret)
>> goto out;
>>
>> - dma_async_issue_pending(csi->dma.chan);
>> + dma_async_issue_pending(ctx->dma.chan);
>>
>> if (!wait_for_completion_timeout(&drain_complete,
>> msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
>> - dmaengine_terminate_sync(csi->dma.chan);
>> + dmaengine_terminate_sync(ctx->dma.chan);
>> dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
>> ret = -ETIMEDOUT;
>> goto out;
>> @@ -596,8 +616,8 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_dev *csi)
>> static void ti_csi2rx_dma_callback(void *param)
>> {
>> struct ti_csi2rx_buffer *buf = param;
>> - struct ti_csi2rx_dev *csi = buf->csi;
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_ctx *ctx = buf->ctx;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> unsigned long flags;
>>
>> /*
>> @@ -605,7 +625,7 @@ static void ti_csi2rx_dma_callback(void *param)
>> * hardware monitor registers.
>> */
>> buf->vb.vb2_buf.timestamp = ktime_get_ns();
>> - buf->vb.sequence = csi->sequence++;
>> + buf->vb.sequence = ctx->sequence++;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>>
>> @@ -617,8 +637,9 @@ static void ti_csi2rx_dma_callback(void *param)
>> while (!list_empty(&dma->queue)) {
>> buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
>>
>> - if (ti_csi2rx_start_dma(csi, buf)) {
>> - dev_err(csi->dev, "Failed to queue the next buffer for DMA\n");
>> + if (ti_csi2rx_start_dma(ctx, buf)) {
>> + dev_err(ctx->csi->dev,
>> + "Failed to queue the next buffer for DMA\n");
> Printing some sort of context identifier would be useful here. Same
> below where applicable.
>
>> vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> } else {
>> list_move_tail(&buf->list, &dma->submitted);
>> @@ -631,17 +652,17 @@ static void ti_csi2rx_dma_callback(void *param)
>> spin_unlock_irqrestore(&dma->lock, flags);
>> }
>>
>> -static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
>> +static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
>> struct ti_csi2rx_buffer *buf)
>> {
>> unsigned long addr;
>> struct dma_async_tx_descriptor *desc;
>> - size_t len = csi->v_fmt.fmt.pix.sizeimage;
>> + size_t len = ctx->v_fmt.fmt.pix.sizeimage;
>> dma_cookie_t cookie;
>> int ret = 0;
>>
>> addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
>> - desc = dmaengine_prep_slave_single(csi->dma.chan, addr, len,
>> + desc = dmaengine_prep_slave_single(ctx->dma.chan, addr, len,
>> DMA_DEV_TO_MEM,
>> DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
>> if (!desc)
>> @@ -655,20 +676,20 @@ static int ti_csi2rx_start_dma(struct ti_csi2rx_dev *csi,
>> if (ret)
>> return ret;
>>
>> - dma_async_issue_pending(csi->dma.chan);
>> + dma_async_issue_pending(ctx->dma.chan);
>>
>> return 0;
>> }
>>
>> -static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
>> +static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
>> {
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> enum ti_csi2rx_dma_state state;
>> unsigned long flags;
>> int ret;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> - state = csi->dma.state;
>> + state = ctx->dma.state;
>> dma->state = TI_CSI2RX_DMA_STOPPED;
>> spin_unlock_irqrestore(&dma->lock, flags);
>>
>> @@ -679,30 +700,30 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_dev *csi)
>> * is stopped, as the module-level pixel reset cannot be
>> * enforced before terminating DMA.
>> */
>> - ret = ti_csi2rx_drain_dma(csi);
>> + ret = ti_csi2rx_drain_dma(ctx);
>> if (ret && ret != -ETIMEDOUT)
>> - dev_warn(csi->dev,
>> + dev_warn(ctx->csi->dev,
>> "Failed to drain DMA. Next frame might be bogus\n");
>> }
>>
>> - ret = dmaengine_terminate_sync(csi->dma.chan);
>> + ret = dmaengine_terminate_sync(ctx->dma.chan);
>> if (ret)
>> - dev_err(csi->dev, "Failed to stop DMA: %d\n", ret);
>> + dev_err(ctx->csi->dev, "Failed to stop DMA: %d\n", ret);
>> }
>>
>> -static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_dev *csi,
>> +static void ti_csi2rx_cleanup_buffers(struct ti_csi2rx_ctx *ctx,
>> enum vb2_buffer_state state)
>> {
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> struct ti_csi2rx_buffer *buf, *tmp;
>> unsigned long flags;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> - list_for_each_entry_safe(buf, tmp, &csi->dma.queue, list) {
>> + list_for_each_entry_safe(buf, tmp, &ctx->dma.queue, list) {
>> list_del(&buf->list);
>> vb2_buffer_done(&buf->vb.vb2_buf, state);
>> }
>> - list_for_each_entry_safe(buf, tmp, &csi->dma.submitted, list) {
>> + list_for_each_entry_safe(buf, tmp, &ctx->dma.submitted, list) {
>> list_del(&buf->list);
>> vb2_buffer_done(&buf->vb.vb2_buf, state);
>> }
>> @@ -713,8 +734,8 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
>> unsigned int *nplanes, unsigned int sizes[],
>> struct device *alloc_devs[])
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(q);
>> - unsigned int size = csi->v_fmt.fmt.pix.sizeimage;
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(q);
>> + unsigned int size = ctx->v_fmt.fmt.pix.sizeimage;
>>
>> if (*nplanes) {
>> if (sizes[0] < size)
>> @@ -730,11 +751,11 @@ static int ti_csi2rx_queue_setup(struct vb2_queue *q, unsigned int *nbuffers,
>>
>> static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
>> - unsigned long size = csi->v_fmt.fmt.pix.sizeimage;
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
>> + unsigned long size = ctx->v_fmt.fmt.pix.sizeimage;
>>
>> if (vb2_plane_size(vb, 0) < size) {
>> - dev_err(csi->dev, "Data will not fit into plane\n");
>> + dev_err(ctx->csi->dev, "Data will not fit into plane\n");
>> return -EINVAL;
>> }
>>
>> @@ -744,15 +765,15 @@ static int ti_csi2rx_buffer_prepare(struct vb2_buffer *vb)
>>
>> static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vb->vb2_queue);
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
>> struct ti_csi2rx_buffer *buf;
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> bool restart_dma = false;
>> unsigned long flags = 0;
>> int ret;
>>
>> buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
>> - buf->csi = csi;
>> + buf->ctx = ctx;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> /*
>> @@ -781,18 +802,18 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
>> * the application and will only confuse it. Issue a DMA
>> * transaction to drain that up.
>> */
>> - ret = ti_csi2rx_drain_dma(csi);
>> + ret = ti_csi2rx_drain_dma(ctx);
>> if (ret && ret != -ETIMEDOUT)
>> - dev_warn(csi->dev,
>> + dev_warn(ctx->csi->dev,
>> "Failed to drain DMA. Next frame might be bogus\n");
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> - ret = ti_csi2rx_start_dma(csi, buf);
>> + ret = ti_csi2rx_start_dma(ctx, buf);
>> if (ret) {
>> vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
>> dma->state = TI_CSI2RX_DMA_IDLE;
>> spin_unlock_irqrestore(&dma->lock, flags);
>> - dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
>> + dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
>> } else {
>> list_add_tail(&buf->list, &dma->submitted);
>> spin_unlock_irqrestore(&dma->lock, flags);
>> @@ -802,8 +823,9 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
>>
>> static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
>> - struct ti_csi2rx_dma *dma = &csi->dma;
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> + struct ti_csi2rx_dma *dma = &ctx->dma;
>> struct ti_csi2rx_buffer *buf;
>> unsigned long flags;
>> int ret = 0;
>> @@ -815,18 +837,18 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> if (ret)
>> return ret;
>>
>> - ret = video_device_pipeline_start(&csi->vdev, &csi->pipe);
>> + ret = video_device_pipeline_start(&ctx->vdev, &csi->pipe);
>> if (ret)
>> goto err;
>>
>> - ti_csi2rx_setup_shim(csi);
>> + ti_csi2rx_setup_shim(ctx);
>>
>> - csi->sequence = 0;
>> + ctx->sequence = 0;
>>
>> spin_lock_irqsave(&dma->lock, flags);
>> buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
>>
>> - ret = ti_csi2rx_start_dma(csi, buf);
>> + ret = ti_csi2rx_start_dma(ctx, buf);
>> if (ret) {
>> dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
>> spin_unlock_irqrestore(&dma->lock, flags);
>> @@ -844,22 +866,23 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> return 0;
>>
>> err_dma:
>> - ti_csi2rx_stop_dma(csi);
>> + ti_csi2rx_stop_dma(ctx);
>> err_pipeline:
>> - video_device_pipeline_stop(&csi->vdev);
>> + video_device_pipeline_stop(&ctx->vdev);
>> writel(0, csi->shim + SHIM_CNTL);
>> writel(0, csi->shim + SHIM_DMACNTX);
>> err:
>> - ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_QUEUED);
>> + ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
>> return ret;
>> }
>>
>> static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
>> {
>> - struct ti_csi2rx_dev *csi = vb2_get_drv_priv(vq);
>> + struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> int ret;
>>
>> - video_device_pipeline_stop(&csi->vdev);
>> + video_device_pipeline_stop(&ctx->vdev);
>>
>> writel(0, csi->shim + SHIM_CNTL);
>> writel(0, csi->shim + SHIM_DMACNTX);
>> @@ -868,8 +891,8 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
>> if (ret)
>> dev_err(csi->dev, "Failed to stop subdev stream\n");
>>
>> - ti_csi2rx_stop_dma(csi);
>> - ti_csi2rx_cleanup_buffers(csi, VB2_BUF_STATE_ERROR);
>> + ti_csi2rx_stop_dma(ctx);
>> + ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_ERROR);
>> }
>>
>> static const struct vb2_ops csi_vb2_qops = {
>> @@ -880,27 +903,50 @@ static const struct vb2_ops csi_vb2_qops = {
>> .stop_streaming = ti_csi2rx_stop_streaming,
>> };
>>
>> -static int ti_csi2rx_init_vb2q(struct ti_csi2rx_dev *csi)
>> +static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
>> {
>> - struct vb2_queue *q = &csi->vidq;
>> + media_device_unregister(&csi->mdev);
>> + v4l2_device_unregister(&csi->v4l2_dev);
>> + media_device_cleanup(&csi->mdev);
>> +}
>> +
>> +static void ti_csi2rx_cleanup_notifier(struct ti_csi2rx_dev *csi)
>> +{
>> + v4l2_async_nf_unregister(&csi->notifier);
>> + v4l2_async_nf_cleanup(&csi->notifier);
>> +}
>> +
>> +static void ti_csi2rx_cleanup_ctx(struct ti_csi2rx_ctx *ctx)
>> +{
>> + dma_release_channel(ctx->dma.chan);
>> + vb2_queue_release(&ctx->vidq);
>> +
>> + video_unregister_device(&ctx->vdev);
>> +
>> + mutex_destroy(&ctx->mutex);
>> +}
>> +
>> +static int ti_csi2rx_init_vb2q(struct ti_csi2rx_ctx *ctx)
>> +{
>> + struct vb2_queue *q = &ctx->vidq;
>> int ret;
>>
>> q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>> q->io_modes = VB2_MMAP | VB2_DMABUF;
>> - q->drv_priv = csi;
>> + q->drv_priv = ctx;
>> q->buf_struct_size = sizeof(struct ti_csi2rx_buffer);
>> q->ops = &csi_vb2_qops;
>> q->mem_ops = &vb2_dma_contig_memops;
>> q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> - q->dev = dmaengine_get_dma_device(csi->dma.chan);
>> - q->lock = &csi->mutex;
>> + q->dev = dmaengine_get_dma_device(ctx->dma.chan);
>> + q->lock = &ctx->mutex;
>> q->min_queued_buffers = 1;
>>
>> ret = vb2_queue_init(q);
>> if (ret)
>> return ret;
>>
>> - csi->vdev.queue = q;
>> + ctx->vdev.queue = q;
>>
>> return 0;
>> }
>> @@ -909,8 +955,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
>> {
>> struct media_entity *entity = link->sink->entity;
>> struct video_device *vdev = media_entity_to_video_device(entity);
>> - struct ti_csi2rx_dev *csi = container_of(vdev, struct ti_csi2rx_dev, vdev);
>> - struct v4l2_pix_format *csi_fmt = &csi->v_fmt.fmt.pix;
>> + struct ti_csi2rx_ctx *ctx = container_of(vdev, struct ti_csi2rx_ctx, vdev);
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> + struct v4l2_pix_format *csi_fmt = &ctx->v_fmt.fmt.pix;
>> struct v4l2_subdev_format source_fmt = {
>> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>> .pad = link->source->index,
>> @@ -963,47 +1010,69 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
>> .link_validate = ti_csi2rx_link_validate,
>> };
>>
>> -static int ti_csi2rx_init_dma(struct ti_csi2rx_dev *csi)
>> +static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
>> {
>> struct dma_slave_config cfg = {
>> .src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
>> };
>> int ret;
>>
>> - INIT_LIST_HEAD(&csi->dma.queue);
>> - INIT_LIST_HEAD(&csi->dma.submitted);
>> - spin_lock_init(&csi->dma.lock);
>> + INIT_LIST_HEAD(&ctx->dma.queue);
>> + INIT_LIST_HEAD(&ctx->dma.submitted);
>> + spin_lock_init(&ctx->dma.lock);
>>
>> - csi->dma.state = TI_CSI2RX_DMA_STOPPED;
>> + ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
>>
>> - csi->dma.chan = dma_request_chan(csi->dev, "rx0");
>> - if (IS_ERR(csi->dma.chan))
>> - return PTR_ERR(csi->dma.chan);
>> + ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
>> + if (IS_ERR(ctx->dma.chan))
>> + return PTR_ERR(ctx->dma.chan);
>>
>> - ret = dmaengine_slave_config(csi->dma.chan, &cfg);
>> + ret = dmaengine_slave_config(ctx->dma.chan, &cfg);
>> if (ret) {
>> - dma_release_channel(csi->dma.chan);
>> + dma_release_channel(ctx->dma.chan);
>> return ret;
>> }
>>
>> - csi->dma.drain.len = DRAIN_BUFFER_SIZE;
>> - csi->dma.drain.vaddr = dma_alloc_coherent(csi->dev, csi->dma.drain.len,
>> - &csi->dma.drain.paddr,
>> - GFP_KERNEL);
>> - if (!csi->dma.drain.vaddr)
>> - return -ENOMEM;
>> -
>> return 0;
>> }
>>
>> static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>> {
>> struct media_device *mdev = &csi->mdev;
>> - struct video_device *vdev = &csi->vdev;
>> + int ret;
>> +
>> + mdev->dev = csi->dev;
>> + mdev->hw_revision = 1;
>> + strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
>> +
>> + media_device_init(mdev);
>> +
>> + csi->v4l2_dev.mdev = mdev;
>> +
>> + ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
>> + if (ret)
>> + return ret;
>> +
>> + ret = media_device_register(mdev);
>> + if (ret) {
>> + v4l2_device_unregister(&csi->v4l2_dev);
>> + media_device_cleanup(mdev);
>> + return ret;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
>> +{
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> + struct video_device *vdev = &ctx->vdev;
>> const struct ti_csi2rx_fmt *fmt;
>> - struct v4l2_pix_format *pix_fmt = &csi->v_fmt.fmt.pix;
>> + struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
>> int ret;
>>
>> + mutex_init(&ctx->mutex);
>> +
>> fmt = find_format_by_fourcc(V4L2_PIX_FMT_UYVY);
>> if (!fmt)
>> return -EINVAL;
>> @@ -1012,19 +1081,20 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>> pix_fmt->height = 480;
>> pix_fmt->field = V4L2_FIELD_NONE;
>> pix_fmt->colorspace = V4L2_COLORSPACE_SRGB;
>> - pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
>> - pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE;
>> - pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB;
>> + pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601,
>> + pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE,
>> + pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB,
>>
>> - ti_csi2rx_fill_fmt(fmt, &csi->v_fmt);
>> + ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
>>
>> - mdev->dev = csi->dev;
>> - mdev->hw_revision = 1;
>> - strscpy(mdev->model, "TI-CSI2RX", sizeof(mdev->model));
>> -
>> - media_device_init(mdev);
>> + csi->pad.flags = MEDIA_PAD_FL_SINK;
>> + vdev->entity.ops = &ti_csi2rx_video_entity_ops;
>> + ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
>> + if (ret)
>> + return ret;
>>
>> - strscpy(vdev->name, TI_CSI2RX_MODULE_NAME, sizeof(vdev->name));
>> + snprintf(vdev->name, sizeof(vdev->name), "%s context %u",
>> + dev_name(csi->dev), ctx->idx);
>> vdev->v4l2_dev = &csi->v4l2_dev;
>> vdev->vfl_dir = VFL_DIR_RX;
>> vdev->fops = &csi_fops;
>> @@ -1032,61 +1102,28 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>> vdev->release = video_device_release_empty;
>> vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
>> V4L2_CAP_IO_MC;
>> - vdev->lock = &csi->mutex;
>> - video_set_drvdata(vdev, csi);
>> + vdev->lock = &ctx->mutex;
>> + video_set_drvdata(vdev, ctx);
>>
>> - csi->pad.flags = MEDIA_PAD_FL_SINK;
>> - vdev->entity.ops = &ti_csi2rx_video_entity_ops;
>> - ret = media_entity_pads_init(&csi->vdev.entity, 1, &csi->pad);
>> + ret = ti_csi2rx_init_dma(ctx);
>> if (ret)
>> return ret;
>>
>> - csi->v4l2_dev.mdev = mdev;
>> -
>> - ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
>> + ret = ti_csi2rx_init_vb2q(ctx);
>> if (ret)
>> - return ret;
>> -
>> - ret = media_device_register(mdev);
>> - if (ret) {
>> - v4l2_device_unregister(&csi->v4l2_dev);
>> - media_device_cleanup(mdev);
>> - return ret;
>> - }
>> + goto cleanup_dma;
>>
>> return 0;
>> -}
>> -
>> -static void ti_csi2rx_cleanup_dma(struct ti_csi2rx_dev *csi)
>> -{
>> - dma_free_coherent(csi->dev, csi->dma.drain.len,
>> - csi->dma.drain.vaddr, csi->dma.drain.paddr);
>> - csi->dma.drain.vaddr = NULL;
>> - dma_release_channel(csi->dma.chan);
>> -}
>> -
>> -static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
>> -{
>> - media_device_unregister(&csi->mdev);
>> - v4l2_device_unregister(&csi->v4l2_dev);
>> - media_device_cleanup(&csi->mdev);
>> -}
>>
>> -static void ti_csi2rx_cleanup_subdev(struct ti_csi2rx_dev *csi)
>> -{
>> - v4l2_async_nf_unregister(&csi->notifier);
>> - v4l2_async_nf_cleanup(&csi->notifier);
>> -}
>> -
>> -static void ti_csi2rx_cleanup_vb2q(struct ti_csi2rx_dev *csi)
>> -{
>> - vb2_queue_release(&csi->vidq);
>> +cleanup_dma:
>> + dma_release_channel(ctx->dma.chan);
>> + return ret;
>> }
>>
>> static int ti_csi2rx_probe(struct platform_device *pdev)
>> {
>> struct ti_csi2rx_dev *csi;
>> - int ret;
>> + int ret, i;
>>
>> csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
>> if (!csi)
>> @@ -1095,62 +1132,69 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
>> csi->dev = &pdev->dev;
>> platform_set_drvdata(pdev, csi);
>>
>> - mutex_init(&csi->mutex);
>> csi->shim = devm_platform_ioremap_resource(pdev, 0);
>> if (IS_ERR(csi->shim)) {
>> ret = PTR_ERR(csi->shim);
>> - goto err_mutex;
>> + return ret;
>> }
>>
>> - ret = ti_csi2rx_init_dma(csi);
>> - if (ret)
>> - goto err_mutex;
>> + csi->drain.len = DRAIN_BUFFER_SIZE;
>> + csi->drain.vaddr = dma_alloc_coherent(csi->dev, csi->drain.len,
>> + &csi->drain.paddr,
>> + GFP_KERNEL);
>> + if (!csi->drain.vaddr)
>> + return -ENOMEM;
>>
>> ret = ti_csi2rx_v4l2_init(csi);
>> - if (ret)
>> - goto err_dma;
>> -
>> - ret = ti_csi2rx_init_vb2q(csi);
>> if (ret)
>> goto err_v4l2;
>>
>> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
>> + csi->ctx[i].idx = i;
>> + csi->ctx[i].csi = csi;
>> + ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
>> + if (ret)
>> + goto err_ctx;
>> + }
>> +
>> ret = ti_csi2rx_notifier_register(csi);
>> if (ret)
>> - goto err_vb2q;
>> + goto err_ctx;
>>
>> ret = of_platform_populate(csi->dev->of_node, NULL, NULL, csi->dev);
>> if (ret) {
>> dev_err(csi->dev, "Failed to create children: %d\n", ret);
>> - goto err_subdev;
>> + goto err_notifier;
>> }
>>
>> return 0;
>>
>> -err_subdev:
>> - ti_csi2rx_cleanup_subdev(csi);
>> -err_vb2q:
>> - ti_csi2rx_cleanup_vb2q(csi);
>> -err_v4l2:
>> +err_notifier:
>> + ti_csi2rx_cleanup_notifier(csi);
>> +err_ctx:
>> + i--;
>> + for (; i >= 0; i--)
>> + ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
>> ti_csi2rx_cleanup_v4l2(csi);
>> -err_dma:
>> - ti_csi2rx_cleanup_dma(csi);
>> -err_mutex:
>> - mutex_destroy(&csi->mutex);
>> +err_v4l2:
>> + dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
>> + csi->drain.paddr);
>> return ret;
>> }
>>
>> static void ti_csi2rx_remove(struct platform_device *pdev)
>> {
>> struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
>> + unsigned int i;
>>
>> - video_unregister_device(&csi->vdev);
>> + for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
>> + ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
>>
>> - ti_csi2rx_cleanup_vb2q(csi);
>> - ti_csi2rx_cleanup_subdev(csi);
>> + ti_csi2rx_cleanup_notifier(csi);
>> ti_csi2rx_cleanup_v4l2(csi);
>> - ti_csi2rx_cleanup_dma(csi);
>>
>> - mutex_destroy(&csi->mutex);
>> + dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
>> + csi->drain.paddr);
>> }
>>
>> static const struct of_device_id ti_csi2rx_of_match[] = {
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 03/13] media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
2025-04-17 6:55 ` [PATCH v3 01/13] dt-bindings: media: ti,j721e-csi2rx-shim: Support 32 dma chans Rishikesh Donadkar
2025-04-17 6:55 ` [PATCH v3 02/13] media: ti: j721e-csi2rx: separate out device and context Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 11:33 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 04/13] media: ti: j721e-csi2rx: allocate DMA channel based on context index Rishikesh Donadkar
` (9 subsequent siblings)
12 siblings, 1 reply; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Pratyush Yadav <p.yadav@ti.com>
Currently the SHIM code to configure the context only touches the first
context. Add support for writing to the context's registers based on the
context index.
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index 36cde2e87aabb..d03dc4e56d306 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -25,7 +25,7 @@
#define SHIM_CNTL 0x10
#define SHIM_CNTL_PIX_RST BIT(0)
-#define SHIM_DMACNTX 0x20
+#define SHIM_DMACNTX(i) (0x20 + ((i) * 0x20))
#define SHIM_DMACNTX_EN BIT(31)
#define SHIM_DMACNTX_YUV422 GENMASK(27, 26)
#define SHIM_DMACNTX_SIZE GENMASK(21, 20)
@@ -35,7 +35,7 @@
#define SHIM_DMACNTX_SIZE_16 1
#define SHIM_DMACNTX_SIZE_32 2
-#define SHIM_PSI_CFG0 0x24
+#define SHIM_PSI_CFG0(i) (0x24 + ((i) * 0x20))
#define SHIM_PSI_CFG0_SRC_TAG GENMASK(15, 0)
#define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
@@ -549,11 +549,11 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
- writel(reg, csi->shim + SHIM_DMACNTX);
+ writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
reg = FIELD_PREP(SHIM_PSI_CFG0_SRC_TAG, 0) |
FIELD_PREP(SHIM_PSI_CFG0_DST_TAG, 0);
- writel(reg, csi->shim + SHIM_PSI_CFG0);
+ writel(reg, csi->shim + SHIM_PSI_CFG0(ctx->idx));
}
static void ti_csi2rx_drain_callback(void *param)
@@ -870,7 +870,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
err_pipeline:
video_device_pipeline_stop(&ctx->vdev);
writel(0, csi->shim + SHIM_CNTL);
- writel(0, csi->shim + SHIM_DMACNTX);
+ writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
err:
ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
return ret;
@@ -885,7 +885,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
video_device_pipeline_stop(&ctx->vdev);
writel(0, csi->shim + SHIM_CNTL);
- writel(0, csi->shim + SHIM_DMACNTX);
+ writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
if (ret)
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 03/13] media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts
2025-04-17 6:55 ` [PATCH v3 03/13] media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts Rishikesh Donadkar
@ 2025-04-21 11:33 ` Laurent Pinchart
0 siblings, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 11:33 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:44PM +0530, Rishikesh Donadkar wrote:
> From: Pratyush Yadav <p.yadav@ti.com>
>
> Currently the SHIM code to configure the context only touches the first
> context. Add support for writing to the context's registers based on the
> context index.
>
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> .../media/platform/ti/j721e-csi2rx/j721e-csi2rx.c | 12 ++++++------
> 1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> index 36cde2e87aabb..d03dc4e56d306 100644
> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> @@ -25,7 +25,7 @@
> #define SHIM_CNTL 0x10
> #define SHIM_CNTL_PIX_RST BIT(0)
>
> -#define SHIM_DMACNTX 0x20
> +#define SHIM_DMACNTX(i) (0x20 + ((i) * 0x20))
> #define SHIM_DMACNTX_EN BIT(31)
> #define SHIM_DMACNTX_YUV422 GENMASK(27, 26)
> #define SHIM_DMACNTX_SIZE GENMASK(21, 20)
> @@ -35,7 +35,7 @@
> #define SHIM_DMACNTX_SIZE_16 1
> #define SHIM_DMACNTX_SIZE_32 2
>
> -#define SHIM_PSI_CFG0 0x24
> +#define SHIM_PSI_CFG0(i) (0x24 + ((i) * 0x20))
> #define SHIM_PSI_CFG0_SRC_TAG GENMASK(15, 0)
> #define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
>
> @@ -549,11 +549,11 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
>
> reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
>
> - writel(reg, csi->shim + SHIM_DMACNTX);
> + writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
>
> reg = FIELD_PREP(SHIM_PSI_CFG0_SRC_TAG, 0) |
> FIELD_PREP(SHIM_PSI_CFG0_DST_TAG, 0);
> - writel(reg, csi->shim + SHIM_PSI_CFG0);
> + writel(reg, csi->shim + SHIM_PSI_CFG0(ctx->idx));
> }
>
> static void ti_csi2rx_drain_callback(void *param)
> @@ -870,7 +870,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
> err_pipeline:
> video_device_pipeline_stop(&ctx->vdev);
> writel(0, csi->shim + SHIM_CNTL);
> - writel(0, csi->shim + SHIM_DMACNTX);
> + writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
> err:
> ti_csi2rx_cleanup_buffers(ctx, VB2_BUF_STATE_QUEUED);
> return ret;
> @@ -885,7 +885,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
> video_device_pipeline_stop(&ctx->vdev);
>
> writel(0, csi->shim + SHIM_CNTL);
> - writel(0, csi->shim + SHIM_DMACNTX);
> + writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
>
> ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
> if (ret)
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 04/13] media: ti: j721e-csi2rx: allocate DMA channel based on context index
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (2 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 03/13] media: ti: j721e-csi2rx: prepare SHIM code for multiple contexts Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 11:36 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 05/13] media: ti: j721e-csi2rx: add a subdev for the core device Rishikesh Donadkar
` (8 subsequent siblings)
12 siblings, 1 reply; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Pratyush Yadav <p.yadav@ti.com>
With multiple contexts, there needs to be a different DMA channel for
each context. Earlier, the DMA channel name was hard coded to "rx0" for
the sake of simplicity. Generate the DMA channel name based on its index
and get the channel corresponding to the context.
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index d03dc4e56d306..523c890139098 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -1015,6 +1015,7 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
struct dma_slave_config cfg = {
.src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
};
+ char name[32];
int ret;
INIT_LIST_HEAD(&ctx->dma.queue);
@@ -1023,7 +1024,8 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
- ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
+ snprintf(name, sizeof(name), "rx%u", ctx->idx);
+ ctx->dma.chan = dma_request_chan(ctx->csi->dev, name);
if (IS_ERR(ctx->dma.chan))
return PTR_ERR(ctx->dma.chan);
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 04/13] media: ti: j721e-csi2rx: allocate DMA channel based on context index
2025-04-17 6:55 ` [PATCH v3 04/13] media: ti: j721e-csi2rx: allocate DMA channel based on context index Rishikesh Donadkar
@ 2025-04-21 11:36 ` Laurent Pinchart
0 siblings, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 11:36 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:45PM +0530, Rishikesh Donadkar wrote:
> From: Pratyush Yadav <p.yadav@ti.com>
>
> With multiple contexts, there needs to be a different DMA channel for
> each context. Earlier, the DMA channel name was hard coded to "rx0" for
> the sake of simplicity. Generate the DMA channel name based on its index
> and get the channel corresponding to the context.
>
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> index d03dc4e56d306..523c890139098 100644
> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> @@ -1015,6 +1015,7 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
> struct dma_slave_config cfg = {
> .src_addr_width = DMA_SLAVE_BUSWIDTH_16_BYTES,
> };
> + char name[32];
That seems a bit long. 5 characters should be enough.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> int ret;
>
> INIT_LIST_HEAD(&ctx->dma.queue);
> @@ -1023,7 +1024,8 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
>
> ctx->dma.state = TI_CSI2RX_DMA_STOPPED;
>
> - ctx->dma.chan = dma_request_chan(ctx->csi->dev, "rx0");
> + snprintf(name, sizeof(name), "rx%u", ctx->idx);
> + ctx->dma.chan = dma_request_chan(ctx->csi->dev, name);
> if (IS_ERR(ctx->dma.chan))
> return PTR_ERR(ctx->dma.chan);
>
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 05/13] media: ti: j721e-csi2rx: add a subdev for the core device
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (3 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 04/13] media: ti: j721e-csi2rx: allocate DMA channel based on context index Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 13:12 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 06/13] media: ti: j721e-csi2rx: get number of contexts from device tree Rishikesh Donadkar
` (7 subsequent siblings)
12 siblings, 1 reply; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Jai Luthra <j-luthra@ti.com>
With single stream capture, it was simpler to use the video device as
the media entity representing the main TI CSI2RX device. Now with multi
stream capture coming into the picture, the model has shifted to each
video device having a link to the main device's subdev. The routing
would then be set on this subdev.
Add this subdev, link each context to this subdev's entity and link the
subdev's entity to the source. Also add an array of media pads. It will
have one sink pad and source pads equal to the number of contexts.
Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 230 ++++++++++++++++--
1 file changed, 205 insertions(+), 25 deletions(-)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index 523c890139098..ea7e331e872af 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -51,6 +51,11 @@
#define MAX_WIDTH_BYTES SZ_16K
#define MAX_HEIGHT_LINES SZ_16K
+#define TI_CSI2RX_PAD_SINK 0
+#define TI_CSI2RX_PAD_FIRST_SOURCE 1
+#define TI_CSI2RX_NUM_SOURCE_PADS 1
+#define TI_CSI2RX_NUM_PADS (1 + TI_CSI2RX_NUM_SOURCE_PADS)
+
#define DRAIN_TIMEOUT_MS 50
#define DRAIN_BUFFER_SIZE SZ_32K
@@ -97,6 +102,7 @@ struct ti_csi2rx_ctx {
struct mutex mutex; /* To serialize ioctls. */
struct v4l2_format v_fmt;
struct ti_csi2rx_dma dma;
+ struct media_pad pad;
u32 sequence;
u32 idx;
};
@@ -104,12 +110,15 @@ struct ti_csi2rx_ctx {
struct ti_csi2rx_dev {
struct device *dev;
void __iomem *shim;
+ struct mutex mutex; /* To serialize ioctls. */
+ unsigned int enable_count;
struct v4l2_device v4l2_dev;
struct media_device mdev;
struct media_pipeline pipe;
- struct media_pad pad;
+ struct media_pad pads[TI_CSI2RX_NUM_PADS];
struct v4l2_async_notifier notifier;
struct v4l2_subdev *source;
+ struct v4l2_subdev subdev;
struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
/* Buffer to drain stale data from PSI-L endpoint */
struct {
@@ -431,6 +440,15 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
int ret, i;
+ /* Create link from source to subdev */
+ ret = v4l2_create_fwnode_links_to_pad(csi->source,
+ &csi->pads[TI_CSI2RX_PAD_SINK],
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret)
+ return ret;
+
+ /* Create and link video nodes for all DMA contexts */
for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
struct video_device *vdev = &ctx->vdev;
@@ -438,13 +456,17 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
if (ret)
goto unregister_dev;
- }
- ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
- MEDIA_LNK_FL_IMMUTABLE |
- MEDIA_LNK_FL_ENABLED);
- if (ret)
- goto unregister_dev;
+ ret = media_create_pad_link(&csi->subdev.entity,
+ TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+ &vdev->entity, 0,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret) {
+ video_unregister_device(vdev);
+ goto unregister_dev;
+ }
+ }
ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
if (ret)
@@ -454,8 +476,10 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
unregister_dev:
i--;
- for (; i >= 0; i--)
+ for (; i >= 0; i--) {
+ media_entity_remove_links(&csi->ctx[i].vdev.entity);
video_unregister_device(&csi->ctx[i].vdev);
+ }
return ret;
}
@@ -859,7 +883,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
dma->state = TI_CSI2RX_DMA_ACTIVE;
spin_unlock_irqrestore(&dma->lock, flags);
- ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+ ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
if (ret)
goto err_dma;
@@ -887,7 +911,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
writel(0, csi->shim + SHIM_CNTL);
writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
- ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+ ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
if (ret)
dev_err(csi->dev, "Failed to stop subdev stream\n");
@@ -903,8 +927,112 @@ static const struct vb2_ops csi_vb2_qops = {
.stop_streaming = ti_csi2rx_stop_streaming,
};
+static inline struct ti_csi2rx_dev *to_csi2rx_dev(struct v4l2_subdev *sd)
+{
+ return container_of(sd, struct ti_csi2rx_dev, subdev);
+}
+
+static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_format *format)
+{
+ struct v4l2_mbus_framefmt *fmt;
+
+ /* No transcoding, don't allow setting source fmt */
+ if (format->pad > TI_CSI2RX_PAD_SINK)
+ return v4l2_subdev_get_fmt(sd, state, format);
+
+ if (!find_format_by_code(format->format.code))
+ format->format.code = ti_csi2rx_formats[0].code;
+
+ format->format.field = V4L2_FIELD_NONE;
+
+ fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+ *fmt = format->format;
+
+ fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
+ format->stream);
+ *fmt = format->format;
+
+ return 0;
+}
+
+static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state)
+{
+ struct v4l2_subdev_format format = {
+ .pad = TI_CSI2RX_PAD_SINK,
+ .format = {
+ .width = 640,
+ .height = 480,
+ .code = MEDIA_BUS_FMT_UYVY8_1X16,
+ .field = V4L2_FIELD_NONE,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .ycbcr_enc = V4L2_YCBCR_ENC_601,
+ .quantization = V4L2_QUANTIZATION_LIM_RANGE,
+ .xfer_func = V4L2_XFER_FUNC_SRGB,
+ },
+ };
+
+ return ti_csi2rx_sd_set_fmt(sd, state, &format);
+}
+
+static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+ int ret = 0;
+
+ mutex_lock(&csi->mutex);
+
+ if (enable) {
+ if (csi->enable_count > 0) {
+ csi->enable_count++;
+ goto out;
+ }
+
+ ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
+ if (ret)
+ goto out;
+
+ csi->enable_count++;
+ } else {
+ if (csi->enable_count == 0) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (--csi->enable_count > 0)
+ goto out;
+
+ ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+ }
+
+out:
+ mutex_unlock(&csi->mutex);
+ return ret;
+}
+
+static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
+ .get_fmt = v4l2_subdev_get_fmt,
+ .set_fmt = ti_csi2rx_sd_set_fmt,
+};
+
+static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
+ .s_stream = ti_csi2rx_sd_s_stream,
+};
+
+static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
+ .video = &ti_csi2rx_subdev_video_ops,
+ .pad = &ti_csi2rx_subdev_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops ti_csi2rx_internal_ops = {
+ .init_state = ti_csi2rx_sd_init_state,
+};
+
static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
{
+ v4l2_subdev_cleanup(&csi->subdev);
media_device_unregister(&csi->mdev);
v4l2_device_unregister(&csi->v4l2_dev);
media_device_cleanup(&csi->mdev);
@@ -961,14 +1089,22 @@ static int ti_csi2rx_link_validate(struct media_link *link)
struct v4l2_subdev_format source_fmt = {
.which = V4L2_SUBDEV_FORMAT_ACTIVE,
.pad = link->source->index,
+ .stream = 0,
};
+ struct v4l2_subdev_state *state;
const struct ti_csi2rx_fmt *ti_fmt;
int ret;
- ret = v4l2_subdev_call_state_active(csi->source, pad,
- get_fmt, &source_fmt);
- if (ret)
- return ret;
+ state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
+ ret = v4l2_subdev_call(&csi->subdev, pad, get_fmt, state, &source_fmt);
+ v4l2_subdev_unlock_state(state);
+
+ if (ret) {
+ dev_dbg(csi->dev,
+ "Skipping validation as no format present on \"%s\":%u:0\n",
+ link->source->entity->name, link->source->index);
+ return 0;
+ }
if (source_fmt.format.width != csi_fmt->width) {
dev_dbg(csi->dev, "Width does not match (source %u, sink %u)\n",
@@ -998,8 +1134,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
if (ti_fmt->fourcc != csi_fmt->pixelformat) {
dev_dbg(csi->dev,
- "Cannot transform source fmt 0x%x to sink fmt 0x%x\n",
- ti_fmt->fourcc, csi_fmt->pixelformat);
+ "Cannot transform \"%s\":%u format %p4cc to %p4cc\n",
+ link->source->entity->name, link->source->index,
+ &ti_fmt->fourcc, &csi_fmt->pixelformat);
return -EPIPE;
}
@@ -1010,6 +1147,10 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
.link_validate = ti_csi2rx_link_validate,
};
+static const struct media_entity_operations ti_csi2rx_subdev_entity_ops = {
+ .link_validate = v4l2_subdev_link_validate,
+};
+
static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
{
struct dma_slave_config cfg = {
@@ -1041,6 +1182,7 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
{
struct media_device *mdev = &csi->mdev;
+ struct v4l2_subdev *sd = &csi->subdev;
int ret;
mdev->dev = csi->dev;
@@ -1053,16 +1195,51 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
if (ret)
- return ret;
+ goto cleanup_media;
ret = media_device_register(mdev);
- if (ret) {
- v4l2_device_unregister(&csi->v4l2_dev);
- media_device_cleanup(mdev);
- return ret;
- }
+ if (ret)
+ goto unregister_v4l2;
+
+ v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);
+ sd->internal_ops = &ti_csi2rx_internal_ops;
+ sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+ sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
+ sd->dev = csi->dev;
+ sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
+
+ csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
+ for (unsigned int i = TI_CSI2RX_PAD_FIRST_SOURCE;
+ i < TI_CSI2RX_NUM_PADS; i++)
+ csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
+
+ ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+ csi->pads);
+ if (ret)
+ goto unregister_media;
+
+ ret = v4l2_subdev_init_finalize(sd);
+ if (ret)
+ goto unregister_media;
+
+ ret = v4l2_device_register_subdev(&csi->v4l2_dev, sd);
+ if (ret)
+ goto cleanup_subdev;
return 0;
+
+cleanup_subdev:
+ v4l2_subdev_cleanup(sd);
+unregister_media:
+ media_device_unregister(mdev);
+unregister_v4l2:
+ v4l2_device_unregister(&csi->v4l2_dev);
+cleanup_media:
+ media_device_cleanup(mdev);
+
+ return ret;
}
static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
@@ -1089,9 +1266,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
- csi->pad.flags = MEDIA_PAD_FL_SINK;
+ ctx->pad.flags = MEDIA_PAD_FL_SINK;
vdev->entity.ops = &ti_csi2rx_video_entity_ops;
- ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
+ ret = media_entity_pads_init(&ctx->vdev.entity, 1, &ctx->pad);
if (ret)
return ret;
@@ -1147,6 +1324,8 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
if (!csi->drain.vaddr)
return -ENOMEM;
+ mutex_init(&csi->mutex);
+
ret = ti_csi2rx_v4l2_init(csi);
if (ret)
goto err_v4l2;
@@ -1179,6 +1358,7 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
ti_csi2rx_cleanup_v4l2(csi);
err_v4l2:
+ mutex_destroy(&csi->mutex);
dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
csi->drain.paddr);
return ret;
@@ -1194,7 +1374,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
ti_csi2rx_cleanup_notifier(csi);
ti_csi2rx_cleanup_v4l2(csi);
-
+ mutex_destroy(&csi->mutex);
dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
csi->drain.paddr);
}
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 05/13] media: ti: j721e-csi2rx: add a subdev for the core device
2025-04-17 6:55 ` [PATCH v3 05/13] media: ti: j721e-csi2rx: add a subdev for the core device Rishikesh Donadkar
@ 2025-04-21 13:12 ` Laurent Pinchart
2025-05-03 5:54 ` Rishikesh Donadkar
0 siblings, 1 reply; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 13:12 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:46PM +0530, Rishikesh Donadkar wrote:
> From: Jai Luthra <j-luthra@ti.com>
>
> With single stream capture, it was simpler to use the video device as
> the media entity representing the main TI CSI2RX device. Now with multi
> stream capture coming into the picture, the model has shifted to each
> video device having a link to the main device's subdev. The routing
> would then be set on this subdev.
>
> Add this subdev, link each context to this subdev's entity and link the
> subdev's entity to the source. Also add an array of media pads. It will
> have one sink pad and source pads equal to the number of contexts.
>
> Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 230 ++++++++++++++++--
> 1 file changed, 205 insertions(+), 25 deletions(-)
>
> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> index 523c890139098..ea7e331e872af 100644
> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> @@ -51,6 +51,11 @@
> #define MAX_WIDTH_BYTES SZ_16K
> #define MAX_HEIGHT_LINES SZ_16K
>
> +#define TI_CSI2RX_PAD_SINK 0
> +#define TI_CSI2RX_PAD_FIRST_SOURCE 1
> +#define TI_CSI2RX_NUM_SOURCE_PADS 1
> +#define TI_CSI2RX_NUM_PADS (1 + TI_CSI2RX_NUM_SOURCE_PADS)
> +
> #define DRAIN_TIMEOUT_MS 50
> #define DRAIN_BUFFER_SIZE SZ_32K
>
> @@ -97,6 +102,7 @@ struct ti_csi2rx_ctx {
> struct mutex mutex; /* To serialize ioctls. */
> struct v4l2_format v_fmt;
> struct ti_csi2rx_dma dma;
> + struct media_pad pad;
> u32 sequence;
> u32 idx;
> };
> @@ -104,12 +110,15 @@ struct ti_csi2rx_ctx {
> struct ti_csi2rx_dev {
> struct device *dev;
> void __iomem *shim;
> + struct mutex mutex; /* To serialize ioctls. */
> + unsigned int enable_count;
> struct v4l2_device v4l2_dev;
> struct media_device mdev;
> struct media_pipeline pipe;
> - struct media_pad pad;
> + struct media_pad pads[TI_CSI2RX_NUM_PADS];
> struct v4l2_async_notifier notifier;
> struct v4l2_subdev *source;
> + struct v4l2_subdev subdev;
> struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
> /* Buffer to drain stale data from PSI-L endpoint */
> struct {
> @@ -431,6 +440,15 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
> struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
> int ret, i;
>
> + /* Create link from source to subdev */
> + ret = v4l2_create_fwnode_links_to_pad(csi->source,
> + &csi->pads[TI_CSI2RX_PAD_SINK],
> + MEDIA_LNK_FL_IMMUTABLE |
> + MEDIA_LNK_FL_ENABLED);
> + if (ret)
> + return ret;
> +
> + /* Create and link video nodes for all DMA contexts */
> for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
> struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
> struct video_device *vdev = &ctx->vdev;
> @@ -438,13 +456,17 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> if (ret)
> goto unregister_dev;
> - }
>
> - ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
> - MEDIA_LNK_FL_IMMUTABLE |
> - MEDIA_LNK_FL_ENABLED);
> - if (ret)
> - goto unregister_dev;
> + ret = media_create_pad_link(&csi->subdev.entity,
> + TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
> + &vdev->entity, 0,
> + MEDIA_LNK_FL_IMMUTABLE |
> + MEDIA_LNK_FL_ENABLED);
> + if (ret) {
> + video_unregister_device(vdev);
> + goto unregister_dev;
> + }
> + }
>
> ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
> if (ret)
> @@ -454,8 +476,10 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
>
> unregister_dev:
> i--;
> - for (; i >= 0; i--)
> + for (; i >= 0; i--) {
> + media_entity_remove_links(&csi->ctx[i].vdev.entity);
> video_unregister_device(&csi->ctx[i].vdev);
> + }
> return ret;
> }
>
> @@ -859,7 +883,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
> dma->state = TI_CSI2RX_DMA_ACTIVE;
> spin_unlock_irqrestore(&dma->lock, flags);
>
> - ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
> + ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
> if (ret)
> goto err_dma;
>
> @@ -887,7 +911,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
> writel(0, csi->shim + SHIM_CNTL);
> writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
>
> - ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
> + ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
> if (ret)
> dev_err(csi->dev, "Failed to stop subdev stream\n");
>
> @@ -903,8 +927,112 @@ static const struct vb2_ops csi_vb2_qops = {
> .stop_streaming = ti_csi2rx_stop_streaming,
> };
>
> +static inline struct ti_csi2rx_dev *to_csi2rx_dev(struct v4l2_subdev *sd)
> +{
> + return container_of(sd, struct ti_csi2rx_dev, subdev);
> +}
> +
It's customary to place such functions just after the definition of the
corresponding structure.
> +static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_format *format)
> +{
> + struct v4l2_mbus_framefmt *fmt;
> +
> + /* No transcoding, don't allow setting source fmt */
> + if (format->pad > TI_CSI2RX_PAD_SINK)
> + return v4l2_subdev_get_fmt(sd, state, format);
> +
> + if (!find_format_by_code(format->format.code))
> + format->format.code = ti_csi2rx_formats[0].code;
> +
Are there no minimum/maximum limits on the side ?
> + format->format.field = V4L2_FIELD_NONE;
The colorspace fields need handling too. It's typical for userspace to
set them to *_DEFAULT, on the sink pad, and the kernel is supposed then
adjust them as the V4L2 spec indicates that *_DEFAULT can't be returned.
As lots of drivers handle color spaces in a pass-through way, some help
from the V4L2 core would be nice.
> +
> + fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
> + *fmt = format->format;
> +
> + fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
> + format->stream);
> + *fmt = format->format;
> +
> + return 0;
> +}
> +
> +static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state)
> +{
> + struct v4l2_subdev_format format = {
> + .pad = TI_CSI2RX_PAD_SINK,
> + .format = {
> + .width = 640,
> + .height = 480,
> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
> + .field = V4L2_FIELD_NONE,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .ycbcr_enc = V4L2_YCBCR_ENC_601,
> + .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> + .xfer_func = V4L2_XFER_FUNC_SRGB,
> + },
> + };
> +
> + return ti_csi2rx_sd_set_fmt(sd, state, &format);
Given that ti_csi2rx_sd_set_fmt() doesn't really perform any relevant
adjustment for the default format, it may be easier to replace the above
structure with a static const struct v4l2_mbus_framefmt, and assign it
to the sink and source formats. It depends a bit on how
ti_csi2rx_sd_set_fmt() will evolve when adding support for multiple
streams.
> +}
> +
> +static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)
> +{
> + struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
> + int ret = 0;
> +
> + mutex_lock(&csi->mutex);
> +
> + if (enable) {
> + if (csi->enable_count > 0) {
> + csi->enable_count++;
> + goto out;
> + }
> +
> + ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
> + if (ret)
> + goto out;
> +
> + csi->enable_count++;
> + } else {
> + if (csi->enable_count == 0) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + if (--csi->enable_count > 0)
> + goto out;
> +
> + ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
> + }
> +
> +out:
> + mutex_unlock(&csi->mutex);
> + return ret;
> +}
> +
> +static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
Have you run v4l2-compliance ? I would have thought that at least
.enum_mbus_code would be mandatory.
> + .get_fmt = v4l2_subdev_get_fmt,
> + .set_fmt = ti_csi2rx_sd_set_fmt,
> +};
> +
> +static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
> + .s_stream = ti_csi2rx_sd_s_stream,
> +};
> +
> +static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
> + .video = &ti_csi2rx_subdev_video_ops,
> + .pad = &ti_csi2rx_subdev_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops ti_csi2rx_internal_ops = {
> + .init_state = ti_csi2rx_sd_init_state,
> +};
> +
> static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
> {
> + v4l2_subdev_cleanup(&csi->subdev);
> media_device_unregister(&csi->mdev);
> v4l2_device_unregister(&csi->v4l2_dev);
> media_device_cleanup(&csi->mdev);
> @@ -961,14 +1089,22 @@ static int ti_csi2rx_link_validate(struct media_link *link)
> struct v4l2_subdev_format source_fmt = {
> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> .pad = link->source->index,
> + .stream = 0,
> };
> + struct v4l2_subdev_state *state;
> const struct ti_csi2rx_fmt *ti_fmt;
> int ret;
>
> - ret = v4l2_subdev_call_state_active(csi->source, pad,
> - get_fmt, &source_fmt);
> - if (ret)
> - return ret;
> + state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
> + ret = v4l2_subdev_call(&csi->subdev, pad, get_fmt, state, &source_fmt);
> + v4l2_subdev_unlock_state(state);
> +
> + if (ret) {
> + dev_dbg(csi->dev,
> + "Skipping validation as no format present on \"%s\":%u:0\n",
> + link->source->entity->name, link->source->index);
How can no format be present ? Is this for the case where no stream is
routed to a particular context ? If so, the corresponding video device
shouldn't be reached by the pipeline walk algorithm, and this function
shouldn't be called in the first place. Am I missing something ?
> + return 0;
> + }
>
> if (source_fmt.format.width != csi_fmt->width) {
> dev_dbg(csi->dev, "Width does not match (source %u, sink %u)\n",
> @@ -998,8 +1134,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
>
> if (ti_fmt->fourcc != csi_fmt->pixelformat) {
> dev_dbg(csi->dev,
> - "Cannot transform source fmt 0x%x to sink fmt 0x%x\n",
> - ti_fmt->fourcc, csi_fmt->pixelformat);
> + "Cannot transform \"%s\":%u format %p4cc to %p4cc\n",
> + link->source->entity->name, link->source->index,
> + &ti_fmt->fourcc, &csi_fmt->pixelformat);
> return -EPIPE;
> }
>
> @@ -1010,6 +1147,10 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
> .link_validate = ti_csi2rx_link_validate,
> };
>
> +static const struct media_entity_operations ti_csi2rx_subdev_entity_ops = {
> + .link_validate = v4l2_subdev_link_validate,
> +};
> +
> static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
> {
> struct dma_slave_config cfg = {
> @@ -1041,6 +1182,7 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
> static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
> {
> struct media_device *mdev = &csi->mdev;
> + struct v4l2_subdev *sd = &csi->subdev;
> int ret;
>
> mdev->dev = csi->dev;
> @@ -1053,16 +1195,51 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>
> ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
> if (ret)
> - return ret;
> + goto cleanup_media;
>
> ret = media_device_register(mdev);
> - if (ret) {
> - v4l2_device_unregister(&csi->v4l2_dev);
> - media_device_cleanup(mdev);
> - return ret;
> - }
> + if (ret)
> + goto unregister_v4l2;
> +
> + v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);
> + sd->internal_ops = &ti_csi2rx_internal_ops;
> + sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> + sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
> + strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
> + sd->dev = csi->dev;
> + sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
> +
> + csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
> + for (unsigned int i = TI_CSI2RX_PAD_FIRST_SOURCE;
> + i < TI_CSI2RX_NUM_PADS; i++)
> + csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
> +
> + ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
> + csi->pads);
> + if (ret)
> + goto unregister_media;
> +
> + ret = v4l2_subdev_init_finalize(sd);
> + if (ret)
> + goto unregister_media;
> +
> + ret = v4l2_device_register_subdev(&csi->v4l2_dev, sd);
> + if (ret)
> + goto cleanup_subdev;
>
> return 0;
> +
> +cleanup_subdev:
> + v4l2_subdev_cleanup(sd);
> +unregister_media:
> + media_device_unregister(mdev);
> +unregister_v4l2:
> + v4l2_device_unregister(&csi->v4l2_dev);
> +cleanup_media:
> + media_device_cleanup(mdev);
> +
> + return ret;
> }
>
> static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
> @@ -1089,9 +1266,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
>
> ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
>
> - csi->pad.flags = MEDIA_PAD_FL_SINK;
> + ctx->pad.flags = MEDIA_PAD_FL_SINK;
> vdev->entity.ops = &ti_csi2rx_video_entity_ops;
> - ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
> + ret = media_entity_pads_init(&ctx->vdev.entity, 1, &ctx->pad);
> if (ret)
> return ret;
>
> @@ -1147,6 +1324,8 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
> if (!csi->drain.vaddr)
> return -ENOMEM;
>
> + mutex_init(&csi->mutex);
> +
> ret = ti_csi2rx_v4l2_init(csi);
> if (ret)
> goto err_v4l2;
> @@ -1179,6 +1358,7 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
> ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
> ti_csi2rx_cleanup_v4l2(csi);
> err_v4l2:
> + mutex_destroy(&csi->mutex);
> dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
> csi->drain.paddr);
> return ret;
> @@ -1194,7 +1374,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
>
> ti_csi2rx_cleanup_notifier(csi);
> ti_csi2rx_cleanup_v4l2(csi);
> -
> + mutex_destroy(&csi->mutex);
> dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
> csi->drain.paddr);
> }
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 05/13] media: ti: j721e-csi2rx: add a subdev for the core device
2025-04-21 13:12 ` Laurent Pinchart
@ 2025-05-03 5:54 ` Rishikesh Donadkar
0 siblings, 0 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-05-03 5:54 UTC (permalink / raw)
To: Laurent Pinchart
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
On 21/04/25 18:42, Laurent Pinchart wrote:
> Hi Rishikesh,
>
> Thank you for the patch.
Hi Laurent,
Thank you for the review
>
> On Thu, Apr 17, 2025 at 12:25:46PM +0530, Rishikesh Donadkar wrote:
>> From: Jai Luthra <j-luthra@ti.com>
>>
>> With single stream capture, it was simpler to use the video device as
>> the media entity representing the main TI CSI2RX device. Now with multi
>> stream capture coming into the picture, the model has shifted to each
>> video device having a link to the main device's subdev. The routing
>> would then be set on this subdev.
>>
>> Add this subdev, link each context to this subdev's entity and link the
>> subdev's entity to the source. Also add an array of media pads. It will
>> have one sink pad and source pads equal to the number of contexts.
>>
>> Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
>> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
>> Signed-off-by: Jai Luthra <j-luthra@ti.com>
>> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
>> ---
>> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 230 ++++++++++++++++--
>> 1 file changed, 205 insertions(+), 25 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> index 523c890139098..ea7e331e872af 100644
>> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> @@ -51,6 +51,11 @@
>> #define MAX_WIDTH_BYTES SZ_16K
>> #define MAX_HEIGHT_LINES SZ_16K
>>
>> +#define TI_CSI2RX_PAD_SINK 0
>> +#define TI_CSI2RX_PAD_FIRST_SOURCE 1
>> +#define TI_CSI2RX_NUM_SOURCE_PADS 1
>> +#define TI_CSI2RX_NUM_PADS (1 + TI_CSI2RX_NUM_SOURCE_PADS)
>> +
>> #define DRAIN_TIMEOUT_MS 50
>> #define DRAIN_BUFFER_SIZE SZ_32K
>>
>> @@ -97,6 +102,7 @@ struct ti_csi2rx_ctx {
>> struct mutex mutex; /* To serialize ioctls. */
>> struct v4l2_format v_fmt;
>> struct ti_csi2rx_dma dma;
>> + struct media_pad pad;
>> u32 sequence;
>> u32 idx;
>> };
>> @@ -104,12 +110,15 @@ struct ti_csi2rx_ctx {
>> struct ti_csi2rx_dev {
>> struct device *dev;
>> void __iomem *shim;
>> + struct mutex mutex; /* To serialize ioctls. */
>> + unsigned int enable_count;
>> struct v4l2_device v4l2_dev;
>> struct media_device mdev;
>> struct media_pipeline pipe;
>> - struct media_pad pad;
>> + struct media_pad pads[TI_CSI2RX_NUM_PADS];
>> struct v4l2_async_notifier notifier;
>> struct v4l2_subdev *source;
>> + struct v4l2_subdev subdev;
>> struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
>> /* Buffer to drain stale data from PSI-L endpoint */
>> struct {
>> @@ -431,6 +440,15 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
>> struct ti_csi2rx_dev *csi = dev_get_drvdata(notifier->v4l2_dev->dev);
>> int ret, i;
>>
>> + /* Create link from source to subdev */
>> + ret = v4l2_create_fwnode_links_to_pad(csi->source,
>> + &csi->pads[TI_CSI2RX_PAD_SINK],
>> + MEDIA_LNK_FL_IMMUTABLE |
>> + MEDIA_LNK_FL_ENABLED);
>> + if (ret)
>> + return ret;
>> +
>> + /* Create and link video nodes for all DMA contexts */
>> for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
>> struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
>> struct video_device *vdev = &ctx->vdev;
>> @@ -438,13 +456,17 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
>> ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> if (ret)
>> goto unregister_dev;
>> - }
>>
>> - ret = v4l2_create_fwnode_links_to_pad(csi->source, &csi->pad,
>> - MEDIA_LNK_FL_IMMUTABLE |
>> - MEDIA_LNK_FL_ENABLED);
>> - if (ret)
>> - goto unregister_dev;
>> + ret = media_create_pad_link(&csi->subdev.entity,
>> + TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
>> + &vdev->entity, 0,
>> + MEDIA_LNK_FL_IMMUTABLE |
>> + MEDIA_LNK_FL_ENABLED);
>> + if (ret) {
>> + video_unregister_device(vdev);
>> + goto unregister_dev;
>> + }
>> + }
>>
>> ret = v4l2_device_register_subdev_nodes(&csi->v4l2_dev);
>> if (ret)
>> @@ -454,8 +476,10 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
>>
>> unregister_dev:
>> i--;
>> - for (; i >= 0; i--)
>> + for (; i >= 0; i--) {
>> + media_entity_remove_links(&csi->ctx[i].vdev.entity);
>> video_unregister_device(&csi->ctx[i].vdev);
>> + }
>> return ret;
>> }
>>
>> @@ -859,7 +883,7 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> dma->state = TI_CSI2RX_DMA_ACTIVE;
>> spin_unlock_irqrestore(&dma->lock, flags);
>>
>> - ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
>> + ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
>> if (ret)
>> goto err_dma;
>>
>> @@ -887,7 +911,7 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
>> writel(0, csi->shim + SHIM_CNTL);
>> writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
>>
>> - ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
>> + ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
>> if (ret)
>> dev_err(csi->dev, "Failed to stop subdev stream\n");
>>
>> @@ -903,8 +927,112 @@ static const struct vb2_ops csi_vb2_qops = {
>> .stop_streaming = ti_csi2rx_stop_streaming,
>> };
>>
>> +static inline struct ti_csi2rx_dev *to_csi2rx_dev(struct v4l2_subdev *sd)
>> +{
>> + return container_of(sd, struct ti_csi2rx_dev, subdev);
>> +}
>> +
> It's customary to place such functions just after the definition of the
> corresponding structure.
Thanks, Will move it in the next revision.
>
>> +static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_format *format)
>> +{
>> + struct v4l2_mbus_framefmt *fmt;
>> +
>> + /* No transcoding, don't allow setting source fmt */
>> + if (format->pad > TI_CSI2RX_PAD_SINK)
>> + return v4l2_subdev_get_fmt(sd, state, format);
>> +
>> + if (!find_format_by_code(format->format.code))
>> + format->format.code = ti_csi2rx_formats[0].code;
>> +
> Are there no minimum/maximum limits on the side ?
Are you suggesting that same limits as done in ti_csi2rx_fill_fmt()
should be imposed on the width and height?
>
>> + format->format.field = V4L2_FIELD_NONE;
> The colorspace fields need handling too. It's typical for userspace to
> set them to *_DEFAULT, on the sink pad, and the kernel is supposed then
> adjust them as the V4L2 spec indicates that *_DEFAULT can't be returned.
>
> As lots of drivers handle color spaces in a pass-through way, some help
> from the V4L2 core would be nice.
Yes, that would be good to have.
>
>> +
>> + fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
>> + *fmt = format->format;
>> +
>> + fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
>> + format->stream);
>> + *fmt = format->format;
>> +
>> + return 0;
>> +}
>> +
>> +static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
>> + struct v4l2_subdev_state *state)
>> +{
>> + struct v4l2_subdev_format format = {
>> + .pad = TI_CSI2RX_PAD_SINK,
>> + .format = {
>> + .width = 640,
>> + .height = 480,
>> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
>> + .field = V4L2_FIELD_NONE,
>> + .colorspace = V4L2_COLORSPACE_SRGB,
>> + .ycbcr_enc = V4L2_YCBCR_ENC_601,
>> + .quantization = V4L2_QUANTIZATION_LIM_RANGE,
>> + .xfer_func = V4L2_XFER_FUNC_SRGB,
>> + },
>> + };
>> +
>> + return ti_csi2rx_sd_set_fmt(sd, state, &format);
> Given that ti_csi2rx_sd_set_fmt() doesn't really perform any relevant
> adjustment for the default format, it may be easier to replace the above
> structure with a static const struct v4l2_mbus_framefmt, and assign it
> to the sink and source formats. It depends a bit on how
> ti_csi2rx_sd_set_fmt() will evolve when adding support for multiple
> streams.
Right, I will make the changes here to have bisectability. But, the body of
this function will change in [PATCH v3 11/13] of this series.
>
>> +}
>> +
>> +static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)
>> +{
>> + struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
>> + int ret = 0;
>> +
>> + mutex_lock(&csi->mutex);
>> +
>> + if (enable) {
>> + if (csi->enable_count > 0) {
>> + csi->enable_count++;
>> + goto out;
>> + }
>> +
>> + ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
>> + if (ret)
>> + goto out;
>> +
>> + csi->enable_count++;
>> + } else {
>> + if (csi->enable_count == 0) {
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + if (--csi->enable_count > 0)
>> + goto out;
>> +
>> + ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
>> + }
>> +
>> +out:
>> + mutex_unlock(&csi->mutex);
>> + return ret;
>> +}
>> +
>> +static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
> Have you run v4l2-compliance ? I would have thought that at least
> .enum_mbus_code would be mandatory.
Thanks for pointing out, Will add.
>
>> + .get_fmt = v4l2_subdev_get_fmt,
>> + .set_fmt = ti_csi2rx_sd_set_fmt,
>> +};
>> +
>> +static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
>> + .s_stream = ti_csi2rx_sd_s_stream,
>> +};
>> +
>> +static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
>> + .video = &ti_csi2rx_subdev_video_ops,
>> + .pad = &ti_csi2rx_subdev_pad_ops,
>> +};
>> +
>> +static const struct v4l2_subdev_internal_ops ti_csi2rx_internal_ops = {
>> + .init_state = ti_csi2rx_sd_init_state,
>> +};
>> +
>> static void ti_csi2rx_cleanup_v4l2(struct ti_csi2rx_dev *csi)
>> {
>> + v4l2_subdev_cleanup(&csi->subdev);
>> media_device_unregister(&csi->mdev);
>> v4l2_device_unregister(&csi->v4l2_dev);
>> media_device_cleanup(&csi->mdev);
>> @@ -961,14 +1089,22 @@ static int ti_csi2rx_link_validate(struct media_link *link)
>> struct v4l2_subdev_format source_fmt = {
>> .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>> .pad = link->source->index,
>> + .stream = 0,
>> };
>> + struct v4l2_subdev_state *state;
>> const struct ti_csi2rx_fmt *ti_fmt;
>> int ret;
>>
>> - ret = v4l2_subdev_call_state_active(csi->source, pad,
>> - get_fmt, &source_fmt);
>> - if (ret)
>> - return ret;
>> + state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
>> + ret = v4l2_subdev_call(&csi->subdev, pad, get_fmt, state, &source_fmt);
>> + v4l2_subdev_unlock_state(state);
>> +
>> + if (ret) {
>> + dev_dbg(csi->dev,
>> + "Skipping validation as no format present on \"%s\":%u:0\n",
>> + link->source->entity->name, link->source->index);
> How can no format be present ? Is this for the case where no stream is
> routed to a particular context ? If so, the corresponding video device
> shouldn't be reached by the pipeline walk algorithm, and this function
> shouldn't be called in the first place. Am I missing something ?
In this driver we keep all the links enabled and immutable by default.
If no streams are routed to a particular context (i.e the pad is
unused), it wont have
the formats set, in that case we skip validation and return 0. This is
done so that streaming from a context
for which routes and format has been set does not fail..
>
>> + return 0;
>> + }
>>
>> if (source_fmt.format.width != csi_fmt->width) {
>> dev_dbg(csi->dev, "Width does not match (source %u, sink %u)\n",
>> @@ -998,8 +1134,9 @@ static int ti_csi2rx_link_validate(struct media_link *link)
>>
>> if (ti_fmt->fourcc != csi_fmt->pixelformat) {
>> dev_dbg(csi->dev,
>> - "Cannot transform source fmt 0x%x to sink fmt 0x%x\n",
>> - ti_fmt->fourcc, csi_fmt->pixelformat);
>> + "Cannot transform \"%s\":%u format %p4cc to %p4cc\n",
>> + link->source->entity->name, link->source->index,
>> + &ti_fmt->fourcc, &csi_fmt->pixelformat);
>> return -EPIPE;
>> }
>>
>> @@ -1010,6 +1147,10 @@ static const struct media_entity_operations ti_csi2rx_video_entity_ops = {
>> .link_validate = ti_csi2rx_link_validate,
>> };
>>
>> +static const struct media_entity_operations ti_csi2rx_subdev_entity_ops = {
>> + .link_validate = v4l2_subdev_link_validate,
>> +};
>> +
>> static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
>> {
>> struct dma_slave_config cfg = {
>> @@ -1041,6 +1182,7 @@ static int ti_csi2rx_init_dma(struct ti_csi2rx_ctx *ctx)
>> static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>> {
>> struct media_device *mdev = &csi->mdev;
>> + struct v4l2_subdev *sd = &csi->subdev;
>> int ret;
>>
>> mdev->dev = csi->dev;
>> @@ -1053,16 +1195,51 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>>
>> ret = v4l2_device_register(csi->dev, &csi->v4l2_dev);
>> if (ret)
>> - return ret;
>> + goto cleanup_media;
>>
>> ret = media_device_register(mdev);
>> - if (ret) {
>> - v4l2_device_unregister(&csi->v4l2_dev);
>> - media_device_cleanup(mdev);
>> - return ret;
>> - }
>> + if (ret)
>> + goto unregister_v4l2;
>> +
>> + v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);
>> + sd->internal_ops = &ti_csi2rx_internal_ops;
>> + sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
>> + sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
>> + strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
>> + sd->dev = csi->dev;
>> + sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
>> +
>> + csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> +
>> + for (unsigned int i = TI_CSI2RX_PAD_FIRST_SOURCE;
>> + i < TI_CSI2RX_NUM_PADS; i++)
>> + csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
>> +
>> + ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
>> + csi->pads);
>> + if (ret)
>> + goto unregister_media;
>> +
>> + ret = v4l2_subdev_init_finalize(sd);
>> + if (ret)
>> + goto unregister_media;
>> +
>> + ret = v4l2_device_register_subdev(&csi->v4l2_dev, sd);
>> + if (ret)
>> + goto cleanup_subdev;
>>
>> return 0;
>> +
>> +cleanup_subdev:
>> + v4l2_subdev_cleanup(sd);
>> +unregister_media:
>> + media_device_unregister(mdev);
>> +unregister_v4l2:
>> + v4l2_device_unregister(&csi->v4l2_dev);
>> +cleanup_media:
>> + media_device_cleanup(mdev);
>> +
>> + return ret;
>> }
>>
>> static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
>> @@ -1089,9 +1266,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
>>
>> ti_csi2rx_fill_fmt(fmt, &ctx->v_fmt);
>>
>> - csi->pad.flags = MEDIA_PAD_FL_SINK;
>> + ctx->pad.flags = MEDIA_PAD_FL_SINK;
>> vdev->entity.ops = &ti_csi2rx_video_entity_ops;
>> - ret = media_entity_pads_init(&ctx->vdev.entity, 1, &csi->pad);
>> + ret = media_entity_pads_init(&ctx->vdev.entity, 1, &ctx->pad);
>> if (ret)
>> return ret;
>>
>> @@ -1147,6 +1324,8 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
>> if (!csi->drain.vaddr)
>> return -ENOMEM;
>>
>> + mutex_init(&csi->mutex);
>> +
>> ret = ti_csi2rx_v4l2_init(csi);
>> if (ret)
>> goto err_v4l2;
>> @@ -1179,6 +1358,7 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
>> ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
>> ti_csi2rx_cleanup_v4l2(csi);
>> err_v4l2:
>> + mutex_destroy(&csi->mutex);
>> dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
>> csi->drain.paddr);
>> return ret;
>> @@ -1194,7 +1374,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
>>
>> ti_csi2rx_cleanup_notifier(csi);
>> ti_csi2rx_cleanup_v4l2(csi);
>> -
>> + mutex_destroy(&csi->mutex);
>> dma_free_coherent(csi->dev, csi->drain.len, csi->drain.vaddr,
>> csi->drain.paddr);
>> }
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 06/13] media: ti: j721e-csi2rx: get number of contexts from device tree
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (4 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 05/13] media: ti: j721e-csi2rx: add a subdev for the core device Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 13:20 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 07/13] media: cadence: csi2rx: add get_frame_desc wrapper Rishikesh Donadkar
` (6 subsequent siblings)
12 siblings, 1 reply; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Pratyush Yadav <p.yadav@ti.com>
Different platforms that use this driver might have different number of
DMA channels allocated for CSI. So only as many DMA contexts can be used
as the number of DMA channels available. Get the number of channels
provided via device tree and only configure that many contexts, and
hence only that many pads.
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Co-developed-by: Jai Luthra <j-luthra@ti.com>
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 41 ++++++++++++++-----
1 file changed, 30 insertions(+), 11 deletions(-)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index ea7e331e872af..e85d04d7c2ff9 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -40,7 +40,7 @@
#define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
#define PSIL_WORD_SIZE_BYTES 16
-#define TI_CSI2RX_NUM_CTX 1
+#define TI_CSI2RX_MAX_CTX 32
/*
* There are no hard limits on the width or height. The DMA engine can handle
@@ -53,8 +53,8 @@
#define TI_CSI2RX_PAD_SINK 0
#define TI_CSI2RX_PAD_FIRST_SOURCE 1
-#define TI_CSI2RX_NUM_SOURCE_PADS 1
-#define TI_CSI2RX_NUM_PADS (1 + TI_CSI2RX_NUM_SOURCE_PADS)
+#define TI_CSI2RX_MAX_SOURCE_PADS TI_CSI2RX_MAX_CTX
+#define TI_CSI2RX_MAX_PADS (1 + TI_CSI2RX_MAX_SOURCE_PADS)
#define DRAIN_TIMEOUT_MS 50
#define DRAIN_BUFFER_SIZE SZ_32K
@@ -112,14 +112,15 @@ struct ti_csi2rx_dev {
void __iomem *shim;
struct mutex mutex; /* To serialize ioctls. */
unsigned int enable_count;
+ unsigned int num_ctx;
struct v4l2_device v4l2_dev;
struct media_device mdev;
struct media_pipeline pipe;
- struct media_pad pads[TI_CSI2RX_NUM_PADS];
+ struct media_pad pads[TI_CSI2RX_MAX_PADS];
struct v4l2_async_notifier notifier;
struct v4l2_subdev *source;
struct v4l2_subdev subdev;
- struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
+ struct ti_csi2rx_ctx ctx[TI_CSI2RX_MAX_CTX];
/* Buffer to drain stale data from PSI-L endpoint */
struct {
void *vaddr;
@@ -449,7 +450,7 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
return ret;
/* Create and link video nodes for all DMA contexts */
- for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+ for (i = 0; i < csi->num_ctx; i++) {
struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
struct video_device *vdev = &ctx->vdev;
@@ -1212,10 +1213,11 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
for (unsigned int i = TI_CSI2RX_PAD_FIRST_SOURCE;
- i < TI_CSI2RX_NUM_PADS; i++)
+ i < TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx; i++)
csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
- ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
+ ret = media_entity_pads_init(&sd->entity,
+ TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx,
csi->pads);
if (ret)
goto unregister_media;
@@ -1301,8 +1303,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
static int ti_csi2rx_probe(struct platform_device *pdev)
{
+ struct device_node *np = pdev->dev.of_node;
struct ti_csi2rx_dev *csi;
- int ret, i;
+ int ret, i, count;
csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
if (!csi)
@@ -1324,13 +1327,29 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
if (!csi->drain.vaddr)
return -ENOMEM;
+ /* Only use as many contexts as the number of DMA channels allocated. */
+ count = of_property_count_strings(np, "dma-names");
+ if (count < 0) {
+ dev_err(csi->dev, "Failed to get DMA channel count: %d\n",
+ count);
+ return count;
+ }
+
+ csi->num_ctx = count;
+ if (csi->num_ctx > TI_CSI2RX_MAX_CTX) {
+ dev_warn(csi->dev,
+ "%u DMA channels passed. Maximum is %u. Ignoring the rest.\n",
+ csi->num_ctx, TI_CSI2RX_MAX_CTX);
+ csi->num_ctx = TI_CSI2RX_MAX_CTX;
+ }
+
mutex_init(&csi->mutex);
ret = ti_csi2rx_v4l2_init(csi);
if (ret)
goto err_v4l2;
- for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
+ for (i = 0; i < csi->num_ctx; i++) {
csi->ctx[i].idx = i;
csi->ctx[i].csi = csi;
ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
@@ -1369,7 +1388,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
unsigned int i;
- for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
+ for (i = 0; i < csi->num_ctx; i++)
ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
ti_csi2rx_cleanup_notifier(csi);
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 06/13] media: ti: j721e-csi2rx: get number of contexts from device tree
2025-04-17 6:55 ` [PATCH v3 06/13] media: ti: j721e-csi2rx: get number of contexts from device tree Rishikesh Donadkar
@ 2025-04-21 13:20 ` Laurent Pinchart
2025-05-03 5:59 ` Rishikesh Donadkar
0 siblings, 1 reply; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 13:20 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:47PM +0530, Rishikesh Donadkar wrote:
> From: Pratyush Yadav <p.yadav@ti.com>
>
> Different platforms that use this driver might have different number of
> DMA channels allocated for CSI. So only as many DMA contexts can be used
> as the number of DMA channels available. Get the number of channels
> provided via device tree and only configure that many contexts, and
> hence only that many pads.
>
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> Co-developed-by: Jai Luthra <j-luthra@ti.com>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 41 ++++++++++++++-----
> 1 file changed, 30 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> index ea7e331e872af..e85d04d7c2ff9 100644
> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> @@ -40,7 +40,7 @@
> #define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
>
> #define PSIL_WORD_SIZE_BYTES 16
> -#define TI_CSI2RX_NUM_CTX 1
> +#define TI_CSI2RX_MAX_CTX 32
>
> /*
> * There are no hard limits on the width or height. The DMA engine can handle
> @@ -53,8 +53,8 @@
>
> #define TI_CSI2RX_PAD_SINK 0
> #define TI_CSI2RX_PAD_FIRST_SOURCE 1
> -#define TI_CSI2RX_NUM_SOURCE_PADS 1
> -#define TI_CSI2RX_NUM_PADS (1 + TI_CSI2RX_NUM_SOURCE_PADS)
> +#define TI_CSI2RX_MAX_SOURCE_PADS TI_CSI2RX_MAX_CTX
> +#define TI_CSI2RX_MAX_PADS (1 + TI_CSI2RX_MAX_SOURCE_PADS)
>
> #define DRAIN_TIMEOUT_MS 50
> #define DRAIN_BUFFER_SIZE SZ_32K
> @@ -112,14 +112,15 @@ struct ti_csi2rx_dev {
> void __iomem *shim;
> struct mutex mutex; /* To serialize ioctls. */
> unsigned int enable_count;
> + unsigned int num_ctx;
> struct v4l2_device v4l2_dev;
> struct media_device mdev;
> struct media_pipeline pipe;
> - struct media_pad pads[TI_CSI2RX_NUM_PADS];
> + struct media_pad pads[TI_CSI2RX_MAX_PADS];
> struct v4l2_async_notifier notifier;
> struct v4l2_subdev *source;
> struct v4l2_subdev subdev;
> - struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
> + struct ti_csi2rx_ctx ctx[TI_CSI2RX_MAX_CTX];
> /* Buffer to drain stale data from PSI-L endpoint */
> struct {
> void *vaddr;
> @@ -449,7 +450,7 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
> return ret;
>
> /* Create and link video nodes for all DMA contexts */
> - for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
> + for (i = 0; i < csi->num_ctx; i++) {
> struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
> struct video_device *vdev = &ctx->vdev;
>
> @@ -1212,10 +1213,11 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
> csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>
> for (unsigned int i = TI_CSI2RX_PAD_FIRST_SOURCE;
> - i < TI_CSI2RX_NUM_PADS; i++)
> + i < TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx; i++)
> csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
>
> - ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
> + ret = media_entity_pads_init(&sd->entity,
> + TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx,
> csi->pads);
> if (ret)
> goto unregister_media;
> @@ -1301,8 +1303,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
>
> static int ti_csi2rx_probe(struct platform_device *pdev)
> {
> + struct device_node *np = pdev->dev.of_node;
> struct ti_csi2rx_dev *csi;
> - int ret, i;
> + int ret, i, count;
>
> csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
> if (!csi)
> @@ -1324,13 +1327,29 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
> if (!csi->drain.vaddr)
> return -ENOMEM;
>
> + /* Only use as many contexts as the number of DMA channels allocated. */
> + count = of_property_count_strings(np, "dma-names");
> + if (count < 0) {
> + dev_err(csi->dev, "Failed to get DMA channel count: %d\n",
> + count);
> + return count;
You're leaking the drain buffer.
> + }
> +
> + csi->num_ctx = count;
> + if (csi->num_ctx > TI_CSI2RX_MAX_CTX) {
> + dev_warn(csi->dev,
> + "%u DMA channels passed. Maximum is %u. Ignoring the rest.\n",
> + csi->num_ctx, TI_CSI2RX_MAX_CTX);
I'd rather turn this into a hard error and get the device trees fixed.
> + csi->num_ctx = TI_CSI2RX_MAX_CTX;
> + }
> +
> mutex_init(&csi->mutex);
>
> ret = ti_csi2rx_v4l2_init(csi);
> if (ret)
> goto err_v4l2;
>
> - for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
> + for (i = 0; i < csi->num_ctx; i++) {
> csi->ctx[i].idx = i;
> csi->ctx[i].csi = csi;
> ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
> @@ -1369,7 +1388,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
> struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
> unsigned int i;
>
> - for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
> + for (i = 0; i < csi->num_ctx; i++)
> ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
>
> ti_csi2rx_cleanup_notifier(csi);
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 06/13] media: ti: j721e-csi2rx: get number of contexts from device tree
2025-04-21 13:20 ` Laurent Pinchart
@ 2025-05-03 5:59 ` Rishikesh Donadkar
0 siblings, 0 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-05-03 5:59 UTC (permalink / raw)
To: Laurent Pinchart
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
On 21/04/25 18:50, Laurent Pinchart wrote:
> Hi Rishikesh,
>
> Thank you for the patch.
Hi Laurent,
Thank you for the review !
>
> On Thu, Apr 17, 2025 at 12:25:47PM +0530, Rishikesh Donadkar wrote:
>> From: Pratyush Yadav <p.yadav@ti.com>
>>
>> Different platforms that use this driver might have different number of
>> DMA channels allocated for CSI. So only as many DMA contexts can be used
>> as the number of DMA channels available. Get the number of channels
>> provided via device tree and only configure that many contexts, and
>> hence only that many pads.
>>
>> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
>> Co-developed-by: Jai Luthra <j-luthra@ti.com>
>> Signed-off-by: Jai Luthra <j-luthra@ti.com>
>> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
>> ---
>> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 41 ++++++++++++++-----
>> 1 file changed, 30 insertions(+), 11 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> index ea7e331e872af..e85d04d7c2ff9 100644
>> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> @@ -40,7 +40,7 @@
>> #define SHIM_PSI_CFG0_DST_TAG GENMASK(31, 16)
>>
>> #define PSIL_WORD_SIZE_BYTES 16
>> -#define TI_CSI2RX_NUM_CTX 1
>> +#define TI_CSI2RX_MAX_CTX 32
>>
>> /*
>> * There are no hard limits on the width or height. The DMA engine can handle
>> @@ -53,8 +53,8 @@
>>
>> #define TI_CSI2RX_PAD_SINK 0
>> #define TI_CSI2RX_PAD_FIRST_SOURCE 1
>> -#define TI_CSI2RX_NUM_SOURCE_PADS 1
>> -#define TI_CSI2RX_NUM_PADS (1 + TI_CSI2RX_NUM_SOURCE_PADS)
>> +#define TI_CSI2RX_MAX_SOURCE_PADS TI_CSI2RX_MAX_CTX
>> +#define TI_CSI2RX_MAX_PADS (1 + TI_CSI2RX_MAX_SOURCE_PADS)
>>
>> #define DRAIN_TIMEOUT_MS 50
>> #define DRAIN_BUFFER_SIZE SZ_32K
>> @@ -112,14 +112,15 @@ struct ti_csi2rx_dev {
>> void __iomem *shim;
>> struct mutex mutex; /* To serialize ioctls. */
>> unsigned int enable_count;
>> + unsigned int num_ctx;
>> struct v4l2_device v4l2_dev;
>> struct media_device mdev;
>> struct media_pipeline pipe;
>> - struct media_pad pads[TI_CSI2RX_NUM_PADS];
>> + struct media_pad pads[TI_CSI2RX_MAX_PADS];
>> struct v4l2_async_notifier notifier;
>> struct v4l2_subdev *source;
>> struct v4l2_subdev subdev;
>> - struct ti_csi2rx_ctx ctx[TI_CSI2RX_NUM_CTX];
>> + struct ti_csi2rx_ctx ctx[TI_CSI2RX_MAX_CTX];
>> /* Buffer to drain stale data from PSI-L endpoint */
>> struct {
>> void *vaddr;
>> @@ -449,7 +450,7 @@ static int csi_async_notifier_complete(struct v4l2_async_notifier *notifier)
>> return ret;
>>
>> /* Create and link video nodes for all DMA contexts */
>> - for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
>> + for (i = 0; i < csi->num_ctx; i++) {
>> struct ti_csi2rx_ctx *ctx = &csi->ctx[i];
>> struct video_device *vdev = &ctx->vdev;
>>
>> @@ -1212,10 +1213,11 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
>> csi->pads[TI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>>
>> for (unsigned int i = TI_CSI2RX_PAD_FIRST_SOURCE;
>> - i < TI_CSI2RX_NUM_PADS; i++)
>> + i < TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx; i++)
>> csi->pads[i].flags = MEDIA_PAD_FL_SOURCE;
>>
>> - ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(csi->pads),
>> + ret = media_entity_pads_init(&sd->entity,
>> + TI_CSI2RX_PAD_FIRST_SOURCE + csi->num_ctx,
>> csi->pads);
>> if (ret)
>> goto unregister_media;
>> @@ -1301,8 +1303,9 @@ static int ti_csi2rx_init_ctx(struct ti_csi2rx_ctx *ctx)
>>
>> static int ti_csi2rx_probe(struct platform_device *pdev)
>> {
>> + struct device_node *np = pdev->dev.of_node;
>> struct ti_csi2rx_dev *csi;
>> - int ret, i;
>> + int ret, i, count;
>>
>> csi = devm_kzalloc(&pdev->dev, sizeof(*csi), GFP_KERNEL);
>> if (!csi)
>> @@ -1324,13 +1327,29 @@ static int ti_csi2rx_probe(struct platform_device *pdev)
>> if (!csi->drain.vaddr)
>> return -ENOMEM;
>>
>> + /* Only use as many contexts as the number of DMA channels allocated. */
>> + count = of_property_count_strings(np, "dma-names");
>> + if (count < 0) {
>> + dev_err(csi->dev, "Failed to get DMA channel count: %d\n",
>> + count);
>> + return count;
> You're leaking the drain buffer.
Oops. Will fix in the next revision
>
>> + }
>> +
>> + csi->num_ctx = count;
>> + if (csi->num_ctx > TI_CSI2RX_MAX_CTX) {
>> + dev_warn(csi->dev,
>> + "%u DMA channels passed. Maximum is %u. Ignoring the rest.\n",
>> + csi->num_ctx, TI_CSI2RX_MAX_CTX);
> I'd rather turn this into a hard error and get the device trees fixed.
Right, will do that in the next revision.
>
>> + csi->num_ctx = TI_CSI2RX_MAX_CTX;
>> + }
>> +
>> mutex_init(&csi->mutex);
>>
>> ret = ti_csi2rx_v4l2_init(csi);
>> if (ret)
>> goto err_v4l2;
>>
>> - for (i = 0; i < TI_CSI2RX_NUM_CTX; i++) {
>> + for (i = 0; i < csi->num_ctx; i++) {
>> csi->ctx[i].idx = i;
>> csi->ctx[i].csi = csi;
>> ret = ti_csi2rx_init_ctx(&csi->ctx[i]);
>> @@ -1369,7 +1388,7 @@ static void ti_csi2rx_remove(struct platform_device *pdev)
>> struct ti_csi2rx_dev *csi = platform_get_drvdata(pdev);
>> unsigned int i;
>>
>> - for (i = 0; i < TI_CSI2RX_NUM_CTX; i++)
>> + for (i = 0; i < csi->num_ctx; i++)
>> ti_csi2rx_cleanup_ctx(&csi->ctx[i]);
>>
>> ti_csi2rx_cleanup_notifier(csi);
Regards,
Rishi
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 07/13] media: cadence: csi2rx: add get_frame_desc wrapper
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (5 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 06/13] media: ti: j721e-csi2rx: get number of contexts from device tree Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 7:00 ` 回复: " Changhuang Liang
2025-04-21 13:22 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 08/13] media: ti: j721e-csi2rx: add support for processing virtual channels Rishikesh Donadkar
` (5 subsequent siblings)
12 siblings, 2 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Pratyush Yadav <p.yadav@ti.com>
J721E wrapper CSI2RX driver needs to get the frame descriptor from the
source to find out info about virtual channel. This driver itself does
not touch the routing or virtual channels in any way. So simply pass the
descriptor through from the source.
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
drivers/media/platform/cadence/cdns-csi2rx.c | 24 ++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
index cebcae196eecc..b7e9225f66a41 100644
--- a/drivers/media/platform/cadence/cdns-csi2rx.c
+++ b/drivers/media/platform/cadence/cdns-csi2rx.c
@@ -135,6 +135,21 @@ static const struct csi2rx_fmt *csi2rx_get_fmt_by_code(u32 code)
return NULL;
}
+static int csi2rx_get_frame_desc_from_source(struct csi2rx_priv *csi2rx,
+ struct v4l2_mbus_frame_desc *fd)
+{
+ struct media_pad *remote_pad;
+
+ remote_pad = media_entity_remote_source_pad_unique(&csi2rx->subdev.entity);
+ if (!remote_pad) {
+ dev_err(csi2rx->dev, "No remote pad found for sink\n");
+ return -ENODEV;
+ }
+
+ return v4l2_subdev_call(csi2rx->source_subdev, pad, get_frame_desc,
+ remote_pad->index, fd);
+}
+
static inline
struct csi2rx_priv *v4l2_subdev_to_csi2rx(struct v4l2_subdev *subdev)
{
@@ -458,10 +473,19 @@ static int csi2rx_init_state(struct v4l2_subdev *subdev,
return csi2rx_set_fmt(subdev, state, &format);
}
+static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
+ struct v4l2_mbus_frame_desc *fd)
+{
+ struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+
+ return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+}
+
static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
.enum_mbus_code = csi2rx_enum_mbus_code,
.get_fmt = v4l2_subdev_get_fmt,
.set_fmt = csi2rx_set_fmt,
+ .get_frame_desc = csi2rx_get_frame_desc,
};
static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* 回复: [PATCH v3 07/13] media: cadence: csi2rx: add get_frame_desc wrapper
2025-04-17 6:55 ` [PATCH v3 07/13] media: cadence: csi2rx: add get_frame_desc wrapper Rishikesh Donadkar
@ 2025-04-21 7:00 ` Changhuang Liang
2025-04-21 13:22 ` Laurent Pinchart
1 sibling, 0 replies; 36+ messages in thread
From: Changhuang Liang @ 2025-04-21 7:00 UTC (permalink / raw)
To: Rishikesh Donadkar, jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, Jack Zhu, laurent.pinchart
> From: Pratyush Yadav <p.yadav@ti.com>
>
> J721E wrapper CSI2RX driver needs to get the frame descriptor from the
> source to find out info about virtual channel. This driver itself does not touch
> the routing or virtual channels in any way. So simply pass the descriptor
> through from the source.
>
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
Reviewed-by: Changhuang Liang <changhuang.liang@starfivetech.com>
Best Regards,
Changhuang
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v3 07/13] media: cadence: csi2rx: add get_frame_desc wrapper
2025-04-17 6:55 ` [PATCH v3 07/13] media: cadence: csi2rx: add get_frame_desc wrapper Rishikesh Donadkar
2025-04-21 7:00 ` 回复: " Changhuang Liang
@ 2025-04-21 13:22 ` Laurent Pinchart
1 sibling, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 13:22 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:48PM +0530, Rishikesh Donadkar wrote:
> From: Pratyush Yadav <p.yadav@ti.com>
>
> J721E wrapper CSI2RX driver needs to get the frame descriptor from the
> source to find out info about virtual channel. This driver itself does
> not touch the routing or virtual channels in any way. So simply pass the
> descriptor through from the source.
>
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> drivers/media/platform/cadence/cdns-csi2rx.c | 24 ++++++++++++++++++++
> 1 file changed, 24 insertions(+)
>
> diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
> index cebcae196eecc..b7e9225f66a41 100644
> --- a/drivers/media/platform/cadence/cdns-csi2rx.c
> +++ b/drivers/media/platform/cadence/cdns-csi2rx.c
> @@ -135,6 +135,21 @@ static const struct csi2rx_fmt *csi2rx_get_fmt_by_code(u32 code)
> return NULL;
> }
>
> +static int csi2rx_get_frame_desc_from_source(struct csi2rx_priv *csi2rx,
> + struct v4l2_mbus_frame_desc *fd)
> +{
> + struct media_pad *remote_pad;
> +
> + remote_pad = media_entity_remote_source_pad_unique(&csi2rx->subdev.entity);
> + if (!remote_pad) {
> + dev_err(csi2rx->dev, "No remote pad found for sink\n");
> + return -ENODEV;
> + }
> +
> + return v4l2_subdev_call(csi2rx->source_subdev, pad, get_frame_desc,
> + remote_pad->index, fd);
> +}
> +
> static inline
> struct csi2rx_priv *v4l2_subdev_to_csi2rx(struct v4l2_subdev *subdev)
> {
> @@ -458,10 +473,19 @@ static int csi2rx_init_state(struct v4l2_subdev *subdev,
> return csi2rx_set_fmt(subdev, state, &format);
> }
>
> +static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
> + struct v4l2_mbus_frame_desc *fd)
> +{
> + struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> +
> + return csi2rx_get_frame_desc_from_source(csi2rx, fd);
> +}
> +
> static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
> .enum_mbus_code = csi2rx_enum_mbus_code,
> .get_fmt = v4l2_subdev_get_fmt,
> .set_fmt = csi2rx_set_fmt,
> + .get_frame_desc = csi2rx_get_frame_desc,
> };
>
> static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 08/13] media: ti: j721e-csi2rx: add support for processing virtual channels
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (6 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 07/13] media: cadence: csi2rx: add get_frame_desc wrapper Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 13:34 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 09/13] media: cadence: csi2rx: Use new enable stream APIs Rishikesh Donadkar
` (4 subsequent siblings)
12 siblings, 1 reply; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Jai Luthra <j-luthra@ti.com>
Use get_frame_desc() to get the frame desc from the connected source,
and use the provided virtual channel instead of hardcoded one.
get_frame_desc() works per stream, but as we don't support multiple
streams yet, we will just always use stream 0. If the source doesn't
support get_frame_desc(), fall back to the previous method of always
capturing virtual channel 0.
Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 39 +++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index e85d04d7c2ff9..3e2a0517a9096 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -29,6 +29,7 @@
#define SHIM_DMACNTX_EN BIT(31)
#define SHIM_DMACNTX_YUV422 GENMASK(27, 26)
#define SHIM_DMACNTX_SIZE GENMASK(21, 20)
+#define SHIM_DMACNTX_VC GENMASK(9, 6)
#define SHIM_DMACNTX_FMT GENMASK(5, 0)
#define SHIM_DMACNTX_YUV422_MODE_11 3
#define SHIM_DMACNTX_SIZE_8 0
@@ -105,6 +106,8 @@ struct ti_csi2rx_ctx {
struct media_pad pad;
u32 sequence;
u32 idx;
+ u32 vc;
+ u32 stream;
};
struct ti_csi2rx_dev {
@@ -573,6 +576,7 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
}
reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
+ reg |= FIELD_PREP(SHIM_DMACNTX_VC, ctx->vc);
writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
@@ -846,6 +850,33 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
}
}
+static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
+{
+ struct ti_csi2rx_dev *csi = ctx->csi;
+ struct v4l2_mbus_frame_desc fd;
+ struct media_pad *pad;
+ int ret, i;
+
+ pad = media_entity_remote_pad_unique(&csi->subdev.entity, MEDIA_PAD_FL_SOURCE);
+ if (!pad)
+ return -ENODEV;
+
+ ret = v4l2_subdev_call(csi->source, pad, get_frame_desc, pad->index,
+ &fd);
+ if (ret)
+ return ret;
+
+ if (fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2)
+ return -EINVAL;
+
+ for (i = 0; i < fd.num_entries; i++) {
+ if (ctx->stream == fd.entry[i].stream)
+ return fd.entry[i].bus.csi2.vc;
+ }
+
+ return -ENODEV;
+}
+
static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
{
struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
@@ -866,6 +897,14 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
if (ret)
goto err;
+ ret = ti_csi2rx_get_vc(ctx);
+ if (ret == -ENOIOCTLCMD)
+ ctx->vc = 0;
+ else if (ret < 0)
+ goto err;
+ else
+ ctx->vc = ret;
+
ti_csi2rx_setup_shim(ctx);
ctx->sequence = 0;
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 08/13] media: ti: j721e-csi2rx: add support for processing virtual channels
2025-04-17 6:55 ` [PATCH v3 08/13] media: ti: j721e-csi2rx: add support for processing virtual channels Rishikesh Donadkar
@ 2025-04-21 13:34 ` Laurent Pinchart
2025-05-05 7:06 ` Rishikesh Donadkar
0 siblings, 1 reply; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 13:34 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:49PM +0530, Rishikesh Donadkar wrote:
> From: Jai Luthra <j-luthra@ti.com>
>
> Use get_frame_desc() to get the frame desc from the connected source,
> and use the provided virtual channel instead of hardcoded one.
>
> get_frame_desc() works per stream, but as we don't support multiple
> streams yet, we will just always use stream 0. If the source doesn't
> support get_frame_desc(), fall back to the previous method of always
> capturing virtual channel 0.
>
> Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 39 +++++++++++++++++++
> 1 file changed, 39 insertions(+)
>
> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> index e85d04d7c2ff9..3e2a0517a9096 100644
> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> @@ -29,6 +29,7 @@
> #define SHIM_DMACNTX_EN BIT(31)
> #define SHIM_DMACNTX_YUV422 GENMASK(27, 26)
> #define SHIM_DMACNTX_SIZE GENMASK(21, 20)
> +#define SHIM_DMACNTX_VC GENMASK(9, 6)
> #define SHIM_DMACNTX_FMT GENMASK(5, 0)
> #define SHIM_DMACNTX_YUV422_MODE_11 3
> #define SHIM_DMACNTX_SIZE_8 0
> @@ -105,6 +106,8 @@ struct ti_csi2rx_ctx {
> struct media_pad pad;
> u32 sequence;
> u32 idx;
> + u32 vc;
> + u32 stream;
> };
>
> struct ti_csi2rx_dev {
> @@ -573,6 +576,7 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
> }
>
> reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
> + reg |= FIELD_PREP(SHIM_DMACNTX_VC, ctx->vc);
>
> writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
>
> @@ -846,6 +850,33 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
> }
> }
>
> +static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
> +{
> + struct ti_csi2rx_dev *csi = ctx->csi;
> + struct v4l2_mbus_frame_desc fd;
> + struct media_pad *pad;
> + int ret, i;
i can never be negative, you can make it an unsigned int.
> +
> + pad = media_entity_remote_pad_unique(&csi->subdev.entity, MEDIA_PAD_FL_SOURCE);
> + if (!pad)
> + return -ENODEV;
> +
> + ret = v4l2_subdev_call(csi->source, pad, get_frame_desc, pad->index,
> + &fd);
> + if (ret)
> + return ret;
> +
> + if (fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2)
> + return -EINVAL;
> +
> + for (i = 0; i < fd.num_entries; i++) {
> + if (ctx->stream == fd.entry[i].stream)
> + return fd.entry[i].bus.csi2.vc;
> + }
> +
> + return -ENODEV;
> +}
> +
> static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
> {
> struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
> @@ -866,6 +897,14 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
> if (ret)
> goto err;
>
> + ret = ti_csi2rx_get_vc(ctx);
> + if (ret == -ENOIOCTLCMD)
> + ctx->vc = 0;
> + else if (ret < 0)
> + goto err;
> + else
> + ctx->vc = ret;
> +
When you'll add support for multiple streams in patch 11/13, you will
end up calling .get_frame_desc() once per stream. All calls will return
the same information, so it's a bit wasteful. Would it be possible to
call this function once only at start time, and cache and use the
results for all video devices ?
> ti_csi2rx_setup_shim(ctx);
>
> ctx->sequence = 0;
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 08/13] media: ti: j721e-csi2rx: add support for processing virtual channels
2025-04-21 13:34 ` Laurent Pinchart
@ 2025-05-05 7:06 ` Rishikesh Donadkar
0 siblings, 0 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-05-05 7:06 UTC (permalink / raw)
To: Laurent Pinchart
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
On 21/04/25 19:04, Laurent Pinchart wrote:
> Hi Rishikesh,
>
> Thank you for the patch.
Hi Laurent,
Thanks you for the review.
>
> On Thu, Apr 17, 2025 at 12:25:49PM +0530, Rishikesh Donadkar wrote:
>> From: Jai Luthra <j-luthra@ti.com>
>>
>> Use get_frame_desc() to get the frame desc from the connected source,
>> and use the provided virtual channel instead of hardcoded one.
>>
>> get_frame_desc() works per stream, but as we don't support multiple
>> streams yet, we will just always use stream 0. If the source doesn't
>> support get_frame_desc(), fall back to the previous method of always
>> capturing virtual channel 0.
>>
>> Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
>> Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
>> Signed-off-by: Jai Luthra <j-luthra@ti.com>
>> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
>> ---
>> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 39 +++++++++++++++++++
>> 1 file changed, 39 insertions(+)
>>
>> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> index e85d04d7c2ff9..3e2a0517a9096 100644
>> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
>> @@ -29,6 +29,7 @@
>> #define SHIM_DMACNTX_EN BIT(31)
>> #define SHIM_DMACNTX_YUV422 GENMASK(27, 26)
>> #define SHIM_DMACNTX_SIZE GENMASK(21, 20)
>> +#define SHIM_DMACNTX_VC GENMASK(9, 6)
>> #define SHIM_DMACNTX_FMT GENMASK(5, 0)
>> #define SHIM_DMACNTX_YUV422_MODE_11 3
>> #define SHIM_DMACNTX_SIZE_8 0
>> @@ -105,6 +106,8 @@ struct ti_csi2rx_ctx {
>> struct media_pad pad;
>> u32 sequence;
>> u32 idx;
>> + u32 vc;
>> + u32 stream;
>> };
>>
>> struct ti_csi2rx_dev {
>> @@ -573,6 +576,7 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
>> }
>>
>> reg |= FIELD_PREP(SHIM_DMACNTX_SIZE, fmt->size);
>> + reg |= FIELD_PREP(SHIM_DMACNTX_VC, ctx->vc);
>>
>> writel(reg, csi->shim + SHIM_DMACNTX(ctx->idx));
>>
>> @@ -846,6 +850,33 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
>> }
>> }
>>
>> +static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
>> +{
>> + struct ti_csi2rx_dev *csi = ctx->csi;
>> + struct v4l2_mbus_frame_desc fd;
>> + struct media_pad *pad;
>> + int ret, i;
> i can never be negative, you can make it an unsigned int.
Will fix.
>
>> +
>> + pad = media_entity_remote_pad_unique(&csi->subdev.entity, MEDIA_PAD_FL_SOURCE);
>> + if (!pad)
>> + return -ENODEV;
>> +
>> + ret = v4l2_subdev_call(csi->source, pad, get_frame_desc, pad->index,
>> + &fd);
>> + if (ret)
>> + return ret;
>> +
>> + if (fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2)
>> + return -EINVAL;
>> +
>> + for (i = 0; i < fd.num_entries; i++) {
>> + if (ctx->stream == fd.entry[i].stream)
>> + return fd.entry[i].bus.csi2.vc;
>> + }
>> +
>> + return -ENODEV;
>> +}
>> +
>> static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> {
>> struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
>> @@ -866,6 +897,14 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
>> if (ret)
>> goto err;
>>
>> + ret = ti_csi2rx_get_vc(ctx);
>> + if (ret == -ENOIOCTLCMD)
>> + ctx->vc = 0;
>> + else if (ret < 0)
>> + goto err;
>> + else
>> + ctx->vc = ret;
>> +
> When you'll add support for multiple streams in patch 11/13, you will
> end up calling .get_frame_desc() once per stream. All calls will return
> the same information, so it's a bit wasteful. Would it be possible to
> call this function once only at start time, and cache and use the
> results for all video devices ?
Yes, Right. I will make these changes in 11/13 when adding
muti-stream support.
>
>> ti_csi2rx_setup_shim(ctx);
>>
>> ctx->sequence = 0;
Regards,
Rishikesh
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 09/13] media: cadence: csi2rx: Use new enable stream APIs
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (7 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 08/13] media: ti: j721e-csi2rx: add support for processing virtual channels Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 7:01 ` 回复: " Changhuang Liang
2025-04-21 13:25 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support Rishikesh Donadkar
` (3 subsequent siblings)
12 siblings, 2 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Jai Luthra <j-luthra@ti.com>
The enable_streams() API in v4l2 supports passing a bitmask to enable
each pad/stream combination individually on any media subdev instead of
doing s_stream(1) to start all streams on the subdev at once.
This API is implemented by ds90ub960 driver (FPDLink deser) and thus the
caller (cdns-csi2x) is required to use it. For now we only enable
stream0.
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
drivers/media/platform/cadence/cdns-csi2rx.c | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
index b7e9225f66a41..df05d278df379 100644
--- a/drivers/media/platform/cadence/cdns-csi2rx.c
+++ b/drivers/media/platform/cadence/cdns-csi2rx.c
@@ -224,10 +224,18 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
static int csi2rx_start(struct csi2rx_priv *csi2rx)
{
unsigned int i;
+ struct media_pad *remote_pad;
unsigned long lanes_used = 0;
u32 reg;
int ret;
+ remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+ if (!remote_pad) {
+ dev_err(csi2rx->dev,
+ "Failed to find connected source\n");
+ return -ENODEV;
+ }
+
ret = clk_prepare_enable(csi2rx->p_clk);
if (ret)
return ret;
@@ -311,7 +319,8 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
reset_control_deassert(csi2rx->sys_rst);
- ret = v4l2_subdev_call(csi2rx->source_subdev, video, s_stream, true);
+ ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
+ remote_pad->index, BIT(0));
if (ret)
goto err_disable_sysclk;
@@ -339,6 +348,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
static void csi2rx_stop(struct csi2rx_priv *csi2rx)
{
+ struct media_pad *remote_pad;
unsigned int i;
u32 val;
int ret;
@@ -367,8 +377,12 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
reset_control_assert(csi2rx->p_rst);
clk_disable_unprepare(csi2rx->p_clk);
- if (v4l2_subdev_call(csi2rx->source_subdev, video, s_stream, false))
+ remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+ if (!remote_pad ||
+ v4l2_subdev_disable_streams(csi2rx->source_subdev,
+ remote_pad->index, BIT(0))) {
dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
+ }
if (csi2rx->dphy) {
writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* 回复: [PATCH v3 09/13] media: cadence: csi2rx: Use new enable stream APIs
2025-04-17 6:55 ` [PATCH v3 09/13] media: cadence: csi2rx: Use new enable stream APIs Rishikesh Donadkar
@ 2025-04-21 7:01 ` Changhuang Liang
2025-04-21 13:25 ` Laurent Pinchart
1 sibling, 0 replies; 36+ messages in thread
From: Changhuang Liang @ 2025-04-21 7:01 UTC (permalink / raw)
To: Rishikesh Donadkar, jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, Jack Zhu, laurent.pinchart
> From: Jai Luthra <j-luthra@ti.com>
>
> The enable_streams() API in v4l2 supports passing a bitmask to enable each
> pad/stream combination individually on any media subdev instead of doing
> s_stream(1) to start all streams on the subdev at once.
>
> This API is implemented by ds90ub960 driver (FPDLink deser) and thus the
> caller (cdns-csi2x) is required to use it. For now we only enable stream0.
>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
Reviewed-by: Changhuang Liang <changhuang.liang@starfivetech.com>
Best Regards,
Changhuang
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v3 09/13] media: cadence: csi2rx: Use new enable stream APIs
2025-04-17 6:55 ` [PATCH v3 09/13] media: cadence: csi2rx: Use new enable stream APIs Rishikesh Donadkar
2025-04-21 7:01 ` 回复: " Changhuang Liang
@ 2025-04-21 13:25 ` Laurent Pinchart
1 sibling, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 13:25 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:50PM +0530, Rishikesh Donadkar wrote:
> From: Jai Luthra <j-luthra@ti.com>
>
> The enable_streams() API in v4l2 supports passing a bitmask to enable
> each pad/stream combination individually on any media subdev instead of
> doing s_stream(1) to start all streams on the subdev at once.
>
> This API is implemented by ds90ub960 driver (FPDLink deser) and thus the
> caller (cdns-csi2x) is required to use it. For now we only enable
> stream0.
>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> drivers/media/platform/cadence/cdns-csi2rx.c | 18 ++++++++++++++++--
> 1 file changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
> index b7e9225f66a41..df05d278df379 100644
> --- a/drivers/media/platform/cadence/cdns-csi2rx.c
> +++ b/drivers/media/platform/cadence/cdns-csi2rx.c
> @@ -224,10 +224,18 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
> static int csi2rx_start(struct csi2rx_priv *csi2rx)
> {
> unsigned int i;
> + struct media_pad *remote_pad;
> unsigned long lanes_used = 0;
> u32 reg;
> int ret;
>
> + remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
Can't you use csi2rx->source_pad instead ?
> + if (!remote_pad) {
> + dev_err(csi2rx->dev,
> + "Failed to find connected source\n");
> + return -ENODEV;
> + }
> +
> ret = clk_prepare_enable(csi2rx->p_clk);
> if (ret)
> return ret;
> @@ -311,7 +319,8 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
>
> reset_control_deassert(csi2rx->sys_rst);
>
> - ret = v4l2_subdev_call(csi2rx->source_subdev, video, s_stream, true);
> + ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
> + remote_pad->index, BIT(0));
> if (ret)
> goto err_disable_sysclk;
>
> @@ -339,6 +348,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
>
> static void csi2rx_stop(struct csi2rx_priv *csi2rx)
> {
> + struct media_pad *remote_pad;
> unsigned int i;
> u32 val;
> int ret;
> @@ -367,8 +377,12 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
> reset_control_assert(csi2rx->p_rst);
> clk_disable_unprepare(csi2rx->p_clk);
>
> - if (v4l2_subdev_call(csi2rx->source_subdev, video, s_stream, false))
> + remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> + if (!remote_pad ||
> + v4l2_subdev_disable_streams(csi2rx->source_subdev,
> + remote_pad->index, BIT(0))) {
> dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
> + }
>
> if (csi2rx->dphy) {
> writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (8 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 09/13] media: cadence: csi2rx: Use new enable stream APIs Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-21 7:02 ` 回复: " Changhuang Liang
2025-04-21 13:57 ` Laurent Pinchart
2025-04-17 6:55 ` [PATCH v3 11/13] media: ti: j721e-csi2rx: add multistream support Rishikesh Donadkar
` (2 subsequent siblings)
12 siblings, 2 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Jai Luthra <j-luthra@ti.com>
Cadence CSI-2 bridge IP supports capturing multiple virtual "streams"
of data over the same physical interface using MIPI Virtual Channels.
The V4L2 subdev APIs should reflect this capability and allow per-stream
routing and controls.
While the hardware IP supports usecases where streams coming in the sink
pad can be broadcasted to multiple source pads, the driver will need
significant re-architecture to make that possible. The two users of this
IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both
have only integrated the first source pad i.e stream0 of this IP. So for
now keep it simple and only allow 1-to-1 mapping of streams from sink to
source, without any broadcasting.
With stream routing now supported in the driver, implement the
enable_stream and disable_stream hooks in place of the stream-unaware
s_stream hook.
This allows consumer devices like a DMA bridge or ISP, to enable
particular streams on a source pad, which in turn can be used to enable
only particular streams on the CSI-TX device connected on the sink pad.
Implement a fallback s_stream hook that internally calls enable_stream
on each source pad, for consumer drivers that don't use multi-stream
APIs to still work.
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Co-developed-by: Rishikesh Donadkar <r-donadkar@ti.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
drivers/media/platform/cadence/cdns-csi2rx.c | 378 ++++++++++++++-----
1 file changed, 288 insertions(+), 90 deletions(-)
diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
index df05d278df379..95beb6623de8c 100644
--- a/drivers/media/platform/cadence/cdns-csi2rx.c
+++ b/drivers/media/platform/cadence/cdns-csi2rx.c
@@ -90,6 +90,7 @@ struct csi2rx_priv {
struct reset_control *pixel_rst[CSI2RX_STREAMS_MAX];
struct phy *dphy;
+ u32 vc_select[CSI2RX_STREAMS_MAX];
u8 lanes[CSI2RX_LANES_MAX];
u8 num_lanes;
u8 max_lanes;
@@ -179,29 +180,36 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
{
- struct media_pad *src_pad =
- &csi2rx->source_subdev->entity.pads[csi2rx->source_pad];
+ struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
union phy_configure_opts opts = { };
struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
- struct v4l2_subdev_format sd_fmt = {
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- .pad = CSI2RX_PAD_SINK,
- };
+ struct v4l2_mbus_framefmt *framefmt;
+ struct v4l2_subdev_state *state;
const struct csi2rx_fmt *fmt;
s64 link_freq;
int ret;
- ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
- &sd_fmt);
- if (ret < 0)
- return ret;
-
- fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
+ if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {
+ link_freq = v4l2_get_link_freq(handler, 0, 0);
+ } else {
+ state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
+ framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
+ 0);
+ fmt = csi2rx_get_fmt_by_code(framefmt->code);
+
+ link_freq = v4l2_get_link_freq(handler, fmt->bpp,
+ 2 * csi2rx->num_lanes);
+
+ dev_warn(csi2rx->dev,
+ "Guessing link frequency using bitdepth of stream 0.\n");
+ dev_warn(csi2rx->dev,
+ "V4L2_CID_LINK_FREQ control is required for multi format sources.\n");
+ }
- link_freq = v4l2_get_link_freq(src_pad,
- fmt->bpp, 2 * csi2rx->num_lanes);
- if (link_freq < 0)
+ if (link_freq < 0) {
+ dev_err(csi2rx->dev, "Unable to calculate link frequency\n");
return link_freq;
+ }
ret = phy_mipi_dphy_get_default_config_for_hsclk(link_freq,
csi2rx->num_lanes, cfg);
@@ -224,18 +232,10 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
static int csi2rx_start(struct csi2rx_priv *csi2rx)
{
unsigned int i;
- struct media_pad *remote_pad;
unsigned long lanes_used = 0;
u32 reg;
int ret;
- remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
- if (!remote_pad) {
- dev_err(csi2rx->dev,
- "Failed to find connected source\n");
- return -ENODEV;
- }
-
ret = clk_prepare_enable(csi2rx->p_clk);
if (ret)
return ret;
@@ -302,11 +302,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
writel(CSI2RX_STREAM_CFG_FIFO_MODE_LARGE_BUF,
csi2rx->base + CSI2RX_STREAM_CFG_REG(i));
- /*
- * Enable one virtual channel. When multiple virtual channels
- * are supported this will have to be changed.
- */
- writel(CSI2RX_STREAM_DATA_CFG_VC_SELECT(0),
+ writel(csi2rx->vc_select[i],
csi2rx->base + CSI2RX_STREAM_DATA_CFG_REG(i));
writel(CSI2RX_STREAM_CTRL_START,
@@ -319,17 +315,10 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
reset_control_deassert(csi2rx->sys_rst);
- ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
- remote_pad->index, BIT(0));
- if (ret)
- goto err_disable_sysclk;
-
clk_disable_unprepare(csi2rx->p_clk);
return 0;
-err_disable_sysclk:
- clk_disable_unprepare(csi2rx->sys_clk);
err_disable_pixclk:
for (; i > 0; i--) {
reset_control_assert(csi2rx->pixel_rst[i - 1]);
@@ -348,7 +337,6 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
static void csi2rx_stop(struct csi2rx_priv *csi2rx)
{
- struct media_pad *remote_pad;
unsigned int i;
u32 val;
int ret;
@@ -377,13 +365,6 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
reset_control_assert(csi2rx->p_rst);
clk_disable_unprepare(csi2rx->p_clk);
- remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
- if (!remote_pad ||
- v4l2_subdev_disable_streams(csi2rx->source_subdev,
- remote_pad->index, BIT(0))) {
- dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
- }
-
if (csi2rx->dphy) {
writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
@@ -392,37 +373,153 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
}
}
-static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
+static void csi2rx_update_vc_select(struct csi2rx_priv *csi2rx,
+ struct v4l2_subdev_state *state)
+{
+ struct v4l2_mbus_frame_desc fd = {0};
+ struct v4l2_subdev_route *route;
+ unsigned int i;
+ int ret;
+
+ /* Capture VC=0 by default */
+ for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
+ csi2rx->vc_select[i] = CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);
+
+ ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
+ if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
+ dev_dbg(csi2rx->dev,
+ "Failed to get source frame desc, allowing only VC=0\n");
+ return;
+ }
+
+ /* If source provides per-stream VC info, use it to filter by VC */
+ memset(csi2rx->vc_select, 0, sizeof(csi2rx->vc_select));
+
+ for_each_active_route(&state->routing, route) {
+ u32 cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;
+
+ for (i = 0; i < fd.num_entries; i++) {
+ if (fd.entry[i].stream != route->sink_stream)
+ continue;
+
+ csi2rx->vc_select[cdns_stream] |=
+ CSI2RX_STREAM_DATA_CFG_VC_SELECT(fd.entry[i].bus.csi2.vc);
+ }
+ }
+}
+
+static int csi2rx_enable_streams(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
{
struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
- int ret = 0;
+ struct media_pad *remote_pad;
+ u64 sink_streams;
+ int ret;
+
+ remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+ if (!remote_pad) {
+ dev_err(csi2rx->dev,
+ "Failed to find connected source\n");
+ return -ENODEV;
+ }
+
+ sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+ CSI2RX_PAD_SINK,
+ &streams_mask);
mutex_lock(&csi2rx->lock);
+ /*
+ * If we're not the first users, there's no need to
+ * enable the whole controller.
+ */
+ if (!csi2rx->count) {
+ ret = csi2rx_start(csi2rx);
+ if (ret)
+ goto err_stream_start;
+ }
- if (enable) {
- /*
- * If we're not the first users, there's no need to
- * enable the whole controller.
- */
- if (!csi2rx->count) {
- ret = csi2rx_start(csi2rx);
- if (ret)
- goto out;
- }
+ /* Start streaming on the source */
+ ret = v4l2_subdev_enable_streams(csi2rx->source_subdev, remote_pad->index,
+ sink_streams);
+ if (ret) {
+ dev_err(csi2rx->dev,
+ "Failed to start streams %#llx on subdev\n",
+ sink_streams);
+ goto err_subdev_enable;
+ }
- csi2rx->count++;
- } else {
- csi2rx->count--;
+ csi2rx->count++;
+ mutex_unlock(&csi2rx->lock);
+
+ return 0;
+
+err_subdev_enable:
+ if (!csi2rx->count)
+ csi2rx_stop(csi2rx);
+err_stream_start:
+ mutex_unlock(&csi2rx->lock);
+ return ret;
+}
+
+static int csi2rx_disable_streams(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+ struct media_pad *remote_pad;
+ u64 sink_streams;
+
+ sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+ CSI2RX_PAD_SINK,
+ &streams_mask);
- /*
- * Let the last user turn off the lights.
- */
- if (!csi2rx->count)
- csi2rx_stop(csi2rx);
+ remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
+ if (!remote_pad ||
+ v4l2_subdev_disable_streams(csi2rx->source_subdev,
+ remote_pad->index, sink_streams)) {
+ dev_err(csi2rx->dev, "Couldn't disable our subdev\n");
}
-out:
+ mutex_lock(&csi2rx->lock);
+ csi2rx->count--;
+ /*
+ * Let the last user turn off the lights.
+ */
+ if (!csi2rx->count)
+ csi2rx_stop(csi2rx);
mutex_unlock(&csi2rx->lock);
+
+ return 0;
+}
+
+static int csi2rx_s_stream_fallback(struct v4l2_subdev *sd, int enable)
+{
+ struct v4l2_subdev_state *state;
+ struct v4l2_subdev_route *route;
+ u64 mask[CSI2RX_PAD_MAX] = {0};
+ int i, ret;
+
+ /* Find the stream mask on all source pads */
+ state = v4l2_subdev_lock_and_get_active_state(sd);
+ for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+ for_each_active_route(&state->routing, route) {
+ if (route->source_pad == i)
+ mask[i] |= BIT_ULL(route->source_stream);
+ }
+ }
+ v4l2_subdev_unlock_state(state);
+
+ /* Start streaming on each pad */
+ for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
+ if (enable)
+ ret = v4l2_subdev_enable_streams(sd, i, mask[i]);
+ else
+ ret = v4l2_subdev_disable_streams(sd, i, mask[i]);
+ if (ret)
+ return ret;
+ }
+
return ret;
}
@@ -438,12 +535,58 @@ static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
return 0;
}
+static int _csi2rx_set_routing(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_krouting *routing)
+{
+ static const struct v4l2_mbus_framefmt format = {
+ .width = 640,
+ .height = 480,
+ .code = MEDIA_BUS_FMT_UYVY8_1X16,
+ .field = V4L2_FIELD_NONE,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .ycbcr_enc = V4L2_YCBCR_ENC_601,
+ .quantization = V4L2_QUANTIZATION_LIM_RANGE,
+ .xfer_func = V4L2_XFER_FUNC_SRGB,
+ };
+ int ret;
+
+ if (routing->num_routes > V4L2_FRAME_DESC_ENTRY_MAX)
+ return -EINVAL;
+
+ ret = v4l2_subdev_routing_validate(subdev, routing,
+ V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
+ if (ret)
+ return ret;
+
+ return v4l2_subdev_set_routing_with_fmt(subdev, state, routing, &format);
+}
+
+static int csi2rx_set_routing(struct v4l2_subdev *subdev,
+ struct v4l2_subdev_state *state,
+ enum v4l2_subdev_format_whence which,
+ struct v4l2_subdev_krouting *routing)
+{
+ struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+ int ret;
+
+ if (which == V4L2_SUBDEV_FORMAT_ACTIVE && csi2rx->count)
+ return -EBUSY;
+
+ ret = _csi2rx_set_routing(subdev, state, routing);
+ if (ret)
+ return ret;
+
+ csi2rx_update_vc_select(csi2rx, state);
+
+ return 0;
+}
+
static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
struct v4l2_subdev_state *state,
struct v4l2_subdev_format *format)
{
struct v4l2_mbus_framefmt *fmt;
- unsigned int i;
/* No transcoding, source and sink formats must match. */
if (format->pad != CSI2RX_PAD_SINK)
@@ -455,14 +598,16 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
format->format.field = V4L2_FIELD_NONE;
/* Set sink format */
- fmt = v4l2_subdev_state_get_format(state, format->pad);
+ fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
*fmt = format->format;
- /* Propagate to source formats */
- for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
- fmt = v4l2_subdev_state_get_format(state, i);
- *fmt = format->format;
- }
+ /* Propagate to source format */
+ fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+ format->stream);
+ if (!fmt)
+ return -EINVAL;
+
+ *fmt = format->format;
return 0;
}
@@ -470,40 +615,92 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
static int csi2rx_init_state(struct v4l2_subdev *subdev,
struct v4l2_subdev_state *state)
{
- struct v4l2_subdev_format format = {
- .pad = CSI2RX_PAD_SINK,
- .format = {
- .width = 640,
- .height = 480,
- .code = MEDIA_BUS_FMT_UYVY8_1X16,
- .field = V4L2_FIELD_NONE,
- .colorspace = V4L2_COLORSPACE_SRGB,
- .ycbcr_enc = V4L2_YCBCR_ENC_601,
- .quantization = V4L2_QUANTIZATION_LIM_RANGE,
- .xfer_func = V4L2_XFER_FUNC_SRGB,
+ struct v4l2_subdev_route routes[] = {
+ {
+ .sink_pad = CSI2RX_PAD_SINK,
+ .sink_stream = 0,
+ .source_pad = CSI2RX_PAD_SOURCE_STREAM0,
+ .source_stream = 0,
+ .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
},
};
- return csi2rx_set_fmt(subdev, state, &format);
+ struct v4l2_subdev_krouting routing = {
+ .num_routes = ARRAY_SIZE(routes),
+ .routes = routes,
+ };
+
+ return _csi2rx_set_routing(subdev, state, &routing);
}
static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
struct v4l2_mbus_frame_desc *fd)
{
struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+ struct v4l2_mbus_frame_desc source_fd = {0};
+ struct v4l2_subdev_route *route;
+ struct v4l2_subdev_state *state;
+ int ret;
- return csi2rx_get_frame_desc_from_source(csi2rx, fd);
+ ret = csi2rx_get_frame_desc_from_source(csi2rx, &source_fd);
+ if (ret)
+ return ret;
+
+ fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
+
+ state = v4l2_subdev_lock_and_get_active_state(subdev);
+
+ for_each_active_route(&state->routing, route) {
+ struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
+ unsigned int i;
+
+ if (route->source_pad != pad)
+ continue;
+
+ for (i = 0; i < source_fd.num_entries; i++) {
+ if (source_fd.entry[i].stream == route->sink_stream) {
+ source_entry = &source_fd.entry[i];
+ break;
+ }
+ }
+
+ if (!source_entry) {
+ dev_err(csi2rx->dev,
+ "Failed to find stream from source frame desc\n");
+ ret = -EPIPE;
+ goto err_missing_stream;
+ }
+
+ fd->entry[fd->num_entries].stream = route->source_stream;
+ fd->entry[fd->num_entries].flags = source_entry->flags;
+ fd->entry[fd->num_entries].length = source_entry->length;
+ fd->entry[fd->num_entries].pixelcode = source_entry->pixelcode;
+ fd->entry[fd->num_entries].bus.csi2.vc =
+ source_entry->bus.csi2.vc;
+ fd->entry[fd->num_entries].bus.csi2.dt =
+ source_entry->bus.csi2.dt;
+
+ fd->num_entries++;
+ }
+
+err_missing_stream:
+ v4l2_subdev_unlock_state(state);
+
+ return ret;
}
static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
- .enum_mbus_code = csi2rx_enum_mbus_code,
- .get_fmt = v4l2_subdev_get_fmt,
- .set_fmt = csi2rx_set_fmt,
- .get_frame_desc = csi2rx_get_frame_desc,
+ .enum_mbus_code = csi2rx_enum_mbus_code,
+ .get_fmt = v4l2_subdev_get_fmt,
+ .set_fmt = csi2rx_set_fmt,
+ .get_frame_desc = csi2rx_get_frame_desc,
+ .set_routing = csi2rx_set_routing,
+ .enable_streams = csi2rx_enable_streams,
+ .disable_streams = csi2rx_disable_streams,
};
static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
- .s_stream = csi2rx_s_stream,
+ .s_stream = csi2rx_s_stream_fallback,
};
static const struct v4l2_subdev_ops csi2rx_subdev_ops = {
@@ -735,7 +932,8 @@ static int csi2rx_probe(struct platform_device *pdev)
csi2rx->pads[CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++)
csi2rx->pads[i].flags = MEDIA_PAD_FL_SOURCE;
- csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+ csi2rx->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+ V4L2_SUBDEV_FL_STREAMS;
csi2rx->subdev.entity.ops = &csi2rx_media_ops;
ret = media_entity_pads_init(&csi2rx->subdev.entity, CSI2RX_PAD_MAX,
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* 回复: [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support
2025-04-17 6:55 ` [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support Rishikesh Donadkar
@ 2025-04-21 7:02 ` Changhuang Liang
2025-04-21 13:57 ` Laurent Pinchart
1 sibling, 0 replies; 36+ messages in thread
From: Changhuang Liang @ 2025-04-21 7:02 UTC (permalink / raw)
To: Rishikesh Donadkar, jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, Jack Zhu, laurent.pinchart
> From: Jai Luthra <j-luthra@ti.com>
>
> Cadence CSI-2 bridge IP supports capturing multiple virtual "streams"
> of data over the same physical interface using MIPI Virtual Channels.
>
> The V4L2 subdev APIs should reflect this capability and allow per-stream
> routing and controls.
>
> While the hardware IP supports usecases where streams coming in the sink
> pad can be broadcasted to multiple source pads, the driver will need
> significant re-architecture to make that possible. The two users of this IP in
> mainline linux are TI Shim and StarFive JH7110 CAMSS, and both have only
> integrated the first source pad i.e stream0 of this IP. So for now keep it simple
> and only allow 1-to-1 mapping of streams from sink to source, without any
> broadcasting.
>
> With stream routing now supported in the driver, implement the
> enable_stream and disable_stream hooks in place of the stream-unaware
> s_stream hook.
>
> This allows consumer devices like a DMA bridge or ISP, to enable particular
> streams on a source pad, which in turn can be used to enable only particular
> streams on the CSI-TX device connected on the sink pad.
>
> Implement a fallback s_stream hook that internally calls enable_stream on
> each source pad, for consumer drivers that don't use multi-stream APIs to still
> work.
>
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Co-developed-by: Rishikesh Donadkar <r-donadkar@ti.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
Reviewed-by: Changhuang Liang <changhuang.liang@starfivetech.com>
Best Regards,
Changhuang
^ permalink raw reply [flat|nested] 36+ messages in thread
* Re: [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support
2025-04-17 6:55 ` [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support Rishikesh Donadkar
2025-04-21 7:02 ` 回复: " Changhuang Liang
@ 2025-04-21 13:57 ` Laurent Pinchart
2025-05-06 11:06 ` Rishikesh Donadkar
1 sibling, 1 reply; 36+ messages in thread
From: Laurent Pinchart @ 2025-04-21 13:57 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
Thank you for the patch.
On Thu, Apr 17, 2025 at 12:25:51PM +0530, Rishikesh Donadkar wrote:
> From: Jai Luthra <j-luthra@ti.com>
>
> Cadence CSI-2 bridge IP supports capturing multiple virtual "streams"
> of data over the same physical interface using MIPI Virtual Channels.
>
> The V4L2 subdev APIs should reflect this capability and allow per-stream
> routing and controls.
>
> While the hardware IP supports usecases where streams coming in the sink
> pad can be broadcasted to multiple source pads, the driver will need
> significant re-architecture to make that possible. The two users of this
> IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both
> have only integrated the first source pad i.e stream0 of this IP. So for
> now keep it simple and only allow 1-to-1 mapping of streams from sink to
> source, without any broadcasting.
>
> With stream routing now supported in the driver, implement the
> enable_stream and disable_stream hooks in place of the stream-unaware
> s_stream hook.
>
> This allows consumer devices like a DMA bridge or ISP, to enable
> particular streams on a source pad, which in turn can be used to enable
> only particular streams on the CSI-TX device connected on the sink pad.
>
> Implement a fallback s_stream hook that internally calls enable_stream
> on each source pad, for consumer drivers that don't use multi-stream
> APIs to still work.
Can't you use v4l2_subdev_s_stream_helper() ? If not, please briefly
explain why in the commit message.
> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> Co-developed-by: Rishikesh Donadkar <r-donadkar@ti.com>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> drivers/media/platform/cadence/cdns-csi2rx.c | 378 ++++++++++++++-----
> 1 file changed, 288 insertions(+), 90 deletions(-)
>
> diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
> index df05d278df379..95beb6623de8c 100644
> --- a/drivers/media/platform/cadence/cdns-csi2rx.c
> +++ b/drivers/media/platform/cadence/cdns-csi2rx.c
> @@ -90,6 +90,7 @@ struct csi2rx_priv {
> struct reset_control *pixel_rst[CSI2RX_STREAMS_MAX];
> struct phy *dphy;
>
> + u32 vc_select[CSI2RX_STREAMS_MAX];
> u8 lanes[CSI2RX_LANES_MAX];
> u8 num_lanes;
> u8 max_lanes;
> @@ -179,29 +180,36 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
>
> static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
> {
> - struct media_pad *src_pad =
> - &csi2rx->source_subdev->entity.pads[csi2rx->source_pad];
> + struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
> union phy_configure_opts opts = { };
> struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
> - struct v4l2_subdev_format sd_fmt = {
> - .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> - .pad = CSI2RX_PAD_SINK,
> - };
> + struct v4l2_mbus_framefmt *framefmt;
> + struct v4l2_subdev_state *state;
> const struct csi2rx_fmt *fmt;
> s64 link_freq;
> int ret;
>
> - ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
> - &sd_fmt);
> - if (ret < 0)
> - return ret;
> -
> - fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
> + if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {
> + link_freq = v4l2_get_link_freq(handler, 0, 0);
> + } else {
> + state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
> + framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
> + 0);
> + fmt = csi2rx_get_fmt_by_code(framefmt->code);
> +
> + link_freq = v4l2_get_link_freq(handler, fmt->bpp,
> + 2 * csi2rx->num_lanes);
> +
> + dev_warn(csi2rx->dev,
> + "Guessing link frequency using bitdepth of stream 0.\n");
> + dev_warn(csi2rx->dev,
> + "V4L2_CID_LINK_FREQ control is required for multi format sources.\n");
> + }
Why do you need this complexity, instead of calling the
v4l2_get_link_freq() variant that takes the source pad as its first
argument ?
>
> - link_freq = v4l2_get_link_freq(src_pad,
> - fmt->bpp, 2 * csi2rx->num_lanes);
> - if (link_freq < 0)
> + if (link_freq < 0) {
> + dev_err(csi2rx->dev, "Unable to calculate link frequency\n");
> return link_freq;
> + }
>
> ret = phy_mipi_dphy_get_default_config_for_hsclk(link_freq,
> csi2rx->num_lanes, cfg);
> @@ -224,18 +232,10 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
> static int csi2rx_start(struct csi2rx_priv *csi2rx)
> {
> unsigned int i;
> - struct media_pad *remote_pad;
> unsigned long lanes_used = 0;
> u32 reg;
> int ret;
>
> - remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> - if (!remote_pad) {
> - dev_err(csi2rx->dev,
> - "Failed to find connected source\n");
> - return -ENODEV;
> - }
> -
> ret = clk_prepare_enable(csi2rx->p_clk);
> if (ret)
> return ret;
> @@ -302,11 +302,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
> writel(CSI2RX_STREAM_CFG_FIFO_MODE_LARGE_BUF,
> csi2rx->base + CSI2RX_STREAM_CFG_REG(i));
>
> - /*
> - * Enable one virtual channel. When multiple virtual channels
> - * are supported this will have to be changed.
> - */
> - writel(CSI2RX_STREAM_DATA_CFG_VC_SELECT(0),
> + writel(csi2rx->vc_select[i],
> csi2rx->base + CSI2RX_STREAM_DATA_CFG_REG(i));
>
> writel(CSI2RX_STREAM_CTRL_START,
> @@ -319,17 +315,10 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
>
> reset_control_deassert(csi2rx->sys_rst);
>
> - ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
> - remote_pad->index, BIT(0));
> - if (ret)
> - goto err_disable_sysclk;
> -
> clk_disable_unprepare(csi2rx->p_clk);
>
> return 0;
>
> -err_disable_sysclk:
> - clk_disable_unprepare(csi2rx->sys_clk);
> err_disable_pixclk:
> for (; i > 0; i--) {
> reset_control_assert(csi2rx->pixel_rst[i - 1]);
> @@ -348,7 +337,6 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
>
> static void csi2rx_stop(struct csi2rx_priv *csi2rx)
> {
> - struct media_pad *remote_pad;
> unsigned int i;
> u32 val;
> int ret;
> @@ -377,13 +365,6 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
> reset_control_assert(csi2rx->p_rst);
> clk_disable_unprepare(csi2rx->p_clk);
>
> - remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> - if (!remote_pad ||
> - v4l2_subdev_disable_streams(csi2rx->source_subdev,
> - remote_pad->index, BIT(0))) {
> - dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
> - }
> -
> if (csi2rx->dphy) {
> writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
>
> @@ -392,37 +373,153 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
> }
> }
>
> -static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
> +static void csi2rx_update_vc_select(struct csi2rx_priv *csi2rx,
> + struct v4l2_subdev_state *state)
> +{
> + struct v4l2_mbus_frame_desc fd = {0};
> + struct v4l2_subdev_route *route;
> + unsigned int i;
> + int ret;
> +
> + /* Capture VC=0 by default */
> + for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
> + csi2rx->vc_select[i] = CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);
> +
> + ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
> + if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
> + dev_dbg(csi2rx->dev,
> + "Failed to get source frame desc, allowing only VC=0\n");
> + return;
> + }
> +
> + /* If source provides per-stream VC info, use it to filter by VC */
> + memset(csi2rx->vc_select, 0, sizeof(csi2rx->vc_select));
> +
> + for_each_active_route(&state->routing, route) {
> + u32 cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;
> +
> + for (i = 0; i < fd.num_entries; i++) {
> + if (fd.entry[i].stream != route->sink_stream)
> + continue;
> +
> + csi2rx->vc_select[cdns_stream] |=
> + CSI2RX_STREAM_DATA_CFG_VC_SELECT(fd.entry[i].bus.csi2.vc);
> + }
> + }
> +}
> +
> +static int csi2rx_enable_streams(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask)
> {
> struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> - int ret = 0;
> + struct media_pad *remote_pad;
> + u64 sink_streams;
> + int ret;
> +
> + remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> + if (!remote_pad) {
> + dev_err(csi2rx->dev,
> + "Failed to find connected source\n");
> + return -ENODEV;
> + }
> +
> + sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
> + CSI2RX_PAD_SINK,
> + &streams_mask);
>
> mutex_lock(&csi2rx->lock);
Can you use lock guards ? It will simplify error handling.
> + /*
> + * If we're not the first users, there's no need to
> + * enable the whole controller.
> + */
> + if (!csi2rx->count) {
I would have stored the bitmask of enabled streams instead of a count,
but it doesn't matter that much.
> + ret = csi2rx_start(csi2rx);
> + if (ret)
> + goto err_stream_start;
> + }
>
> - if (enable) {
> - /*
> - * If we're not the first users, there's no need to
> - * enable the whole controller.
> - */
> - if (!csi2rx->count) {
> - ret = csi2rx_start(csi2rx);
> - if (ret)
> - goto out;
> - }
> + /* Start streaming on the source */
> + ret = v4l2_subdev_enable_streams(csi2rx->source_subdev, remote_pad->index,
> + sink_streams);
> + if (ret) {
> + dev_err(csi2rx->dev,
> + "Failed to start streams %#llx on subdev\n",
> + sink_streams);
> + goto err_subdev_enable;
> + }
>
> - csi2rx->count++;
> - } else {
> - csi2rx->count--;
> + csi2rx->count++;
> + mutex_unlock(&csi2rx->lock);
> +
> + return 0;
> +
> +err_subdev_enable:
> + if (!csi2rx->count)
> + csi2rx_stop(csi2rx);
> +err_stream_start:
> + mutex_unlock(&csi2rx->lock);
> + return ret;
> +}
> +
> +static int csi2rx_disable_streams(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state, u32 pad,
> + u64 streams_mask)
> +{
> + struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> + struct media_pad *remote_pad;
> + u64 sink_streams;
> +
> + sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
> + CSI2RX_PAD_SINK,
> + &streams_mask);
>
> - /*
> - * Let the last user turn off the lights.
> - */
> - if (!csi2rx->count)
> - csi2rx_stop(csi2rx);
> + remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> + if (!remote_pad ||
> + v4l2_subdev_disable_streams(csi2rx->source_subdev,
> + remote_pad->index, sink_streams)) {
> + dev_err(csi2rx->dev, "Couldn't disable our subdev\n");
> }
>
> -out:
> + mutex_lock(&csi2rx->lock);
> + csi2rx->count--;
> + /*
> + * Let the last user turn off the lights.
> + */
This can become a single line.
> + if (!csi2rx->count)
> + csi2rx_stop(csi2rx);
> mutex_unlock(&csi2rx->lock);
> +
> + return 0;
> +}
> +
> +static int csi2rx_s_stream_fallback(struct v4l2_subdev *sd, int enable)
> +{
> + struct v4l2_subdev_state *state;
> + struct v4l2_subdev_route *route;
> + u64 mask[CSI2RX_PAD_MAX] = {0};
> + int i, ret;
> +
> + /* Find the stream mask on all source pads */
> + state = v4l2_subdev_lock_and_get_active_state(sd);
> + for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
> + for_each_active_route(&state->routing, route) {
> + if (route->source_pad == i)
> + mask[i] |= BIT_ULL(route->source_stream);
> + }
> + }
> + v4l2_subdev_unlock_state(state);
> +
> + /* Start streaming on each pad */
> + for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
> + if (enable)
> + ret = v4l2_subdev_enable_streams(sd, i, mask[i]);
> + else
> + ret = v4l2_subdev_disable_streams(sd, i, mask[i]);
> + if (ret)
> + return ret;
> + }
> +
> return ret;
> }
>
> @@ -438,12 +535,58 @@ static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
> return 0;
> }
>
> +static int _csi2rx_set_routing(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_krouting *routing)
> +{
> + static const struct v4l2_mbus_framefmt format = {
> + .width = 640,
> + .height = 480,
> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
> + .field = V4L2_FIELD_NONE,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .ycbcr_enc = V4L2_YCBCR_ENC_601,
> + .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> + .xfer_func = V4L2_XFER_FUNC_SRGB,
> + };
> + int ret;
> +
> + if (routing->num_routes > V4L2_FRAME_DESC_ENTRY_MAX)
> + return -EINVAL;
> +
> + ret = v4l2_subdev_routing_validate(subdev, routing,
> + V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
> + if (ret)
> + return ret;
> +
> + return v4l2_subdev_set_routing_with_fmt(subdev, state, routing, &format);
> +}
> +
> +static int csi2rx_set_routing(struct v4l2_subdev *subdev,
> + struct v4l2_subdev_state *state,
> + enum v4l2_subdev_format_whence which,
> + struct v4l2_subdev_krouting *routing)
> +{
> + struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> + int ret;
> +
> + if (which == V4L2_SUBDEV_FORMAT_ACTIVE && csi2rx->count)
> + return -EBUSY;
> +
> + ret = _csi2rx_set_routing(subdev, state, routing);
> + if (ret)
> + return ret;
> +
> + csi2rx_update_vc_select(csi2rx, state);
When setting routing on the TRY state this will update the active state,
which isn't correct. Can you call this function when starting the device
instead ?
> +
> + return 0;
> +}
> +
> static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
> struct v4l2_subdev_state *state,
> struct v4l2_subdev_format *format)
> {
> struct v4l2_mbus_framefmt *fmt;
> - unsigned int i;
>
> /* No transcoding, source and sink formats must match. */
> if (format->pad != CSI2RX_PAD_SINK)
> @@ -455,14 +598,16 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
> format->format.field = V4L2_FIELD_NONE;
>
> /* Set sink format */
> - fmt = v4l2_subdev_state_get_format(state, format->pad);
> + fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
> *fmt = format->format;
>
> - /* Propagate to source formats */
> - for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
> - fmt = v4l2_subdev_state_get_format(state, i);
> - *fmt = format->format;
> - }
> + /* Propagate to source format */
> + fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
> + format->stream);
> + if (!fmt)
> + return -EINVAL;
> +
> + *fmt = format->format;
>
> return 0;
> }
> @@ -470,40 +615,92 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
> static int csi2rx_init_state(struct v4l2_subdev *subdev,
> struct v4l2_subdev_state *state)
> {
> - struct v4l2_subdev_format format = {
> - .pad = CSI2RX_PAD_SINK,
> - .format = {
> - .width = 640,
> - .height = 480,
> - .code = MEDIA_BUS_FMT_UYVY8_1X16,
> - .field = V4L2_FIELD_NONE,
> - .colorspace = V4L2_COLORSPACE_SRGB,
> - .ycbcr_enc = V4L2_YCBCR_ENC_601,
> - .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> - .xfer_func = V4L2_XFER_FUNC_SRGB,
> + struct v4l2_subdev_route routes[] = {
> + {
> + .sink_pad = CSI2RX_PAD_SINK,
> + .sink_stream = 0,
> + .source_pad = CSI2RX_PAD_SOURCE_STREAM0,
> + .source_stream = 0,
> + .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> },
> };
>
> - return csi2rx_set_fmt(subdev, state, &format);
> + struct v4l2_subdev_krouting routing = {
> + .num_routes = ARRAY_SIZE(routes),
> + .routes = routes,
> + };
> +
> + return _csi2rx_set_routing(subdev, state, &routing);
> }
>
> static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
> struct v4l2_mbus_frame_desc *fd)
> {
> struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> + struct v4l2_mbus_frame_desc source_fd = {0};
> + struct v4l2_subdev_route *route;
> + struct v4l2_subdev_state *state;
> + int ret;
>
> - return csi2rx_get_frame_desc_from_source(csi2rx, fd);
> + ret = csi2rx_get_frame_desc_from_source(csi2rx, &source_fd);
> + if (ret)
> + return ret;
> +
> + fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
> +
> + state = v4l2_subdev_lock_and_get_active_state(subdev);
> +
> + for_each_active_route(&state->routing, route) {
> + struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
> + unsigned int i;
> +
> + if (route->source_pad != pad)
> + continue;
> +
> + for (i = 0; i < source_fd.num_entries; i++) {
> + if (source_fd.entry[i].stream == route->sink_stream) {
> + source_entry = &source_fd.entry[i];
> + break;
> + }
> + }
> +
> + if (!source_entry) {
> + dev_err(csi2rx->dev,
> + "Failed to find stream from source frame desc\n");
> + ret = -EPIPE;
> + goto err_missing_stream;
> + }
> +
> + fd->entry[fd->num_entries].stream = route->source_stream;
> + fd->entry[fd->num_entries].flags = source_entry->flags;
> + fd->entry[fd->num_entries].length = source_entry->length;
> + fd->entry[fd->num_entries].pixelcode = source_entry->pixelcode;
> + fd->entry[fd->num_entries].bus.csi2.vc =
> + source_entry->bus.csi2.vc;
> + fd->entry[fd->num_entries].bus.csi2.dt =
> + source_entry->bus.csi2.dt;
> +
> + fd->num_entries++;
> + }
> +
> +err_missing_stream:
> + v4l2_subdev_unlock_state(state);
> +
> + return ret;
> }
>
> static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
> - .enum_mbus_code = csi2rx_enum_mbus_code,
> - .get_fmt = v4l2_subdev_get_fmt,
> - .set_fmt = csi2rx_set_fmt,
> - .get_frame_desc = csi2rx_get_frame_desc,
> + .enum_mbus_code = csi2rx_enum_mbus_code,
> + .get_fmt = v4l2_subdev_get_fmt,
> + .set_fmt = csi2rx_set_fmt,
> + .get_frame_desc = csi2rx_get_frame_desc,
> + .set_routing = csi2rx_set_routing,
> + .enable_streams = csi2rx_enable_streams,
> + .disable_streams = csi2rx_disable_streams,
> };
>
> static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
> - .s_stream = csi2rx_s_stream,
> + .s_stream = csi2rx_s_stream_fallback,
> };
>
> static const struct v4l2_subdev_ops csi2rx_subdev_ops = {
> @@ -735,7 +932,8 @@ static int csi2rx_probe(struct platform_device *pdev)
> csi2rx->pads[CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++)
> csi2rx->pads[i].flags = MEDIA_PAD_FL_SOURCE;
> - csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> + csi2rx->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> + V4L2_SUBDEV_FL_STREAMS;
> csi2rx->subdev.entity.ops = &csi2rx_media_ops;
>
> ret = media_entity_pads_init(&csi2rx->subdev.entity, CSI2RX_PAD_MAX,
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support
2025-04-21 13:57 ` Laurent Pinchart
@ 2025-05-06 11:06 ` Rishikesh Donadkar
2025-05-09 8:47 ` Laurent Pinchart
0 siblings, 1 reply; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-05-06 11:06 UTC (permalink / raw)
To: Laurent Pinchart
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
On 21/04/25 19:27, Laurent Pinchart wrote:
> Hi Rishikesh,
>
> Thank you for the patch.
Hi Laurent,
Thank you for the review !
>
> On Thu, Apr 17, 2025 at 12:25:51PM +0530, Rishikesh Donadkar wrote:
>> From: Jai Luthra <j-luthra@ti.com>
>>
>> Cadence CSI-2 bridge IP supports capturing multiple virtual "streams"
>> of data over the same physical interface using MIPI Virtual Channels.
>>
>> The V4L2 subdev APIs should reflect this capability and allow per-stream
>> routing and controls.
>>
>> While the hardware IP supports usecases where streams coming in the sink
>> pad can be broadcasted to multiple source pads, the driver will need
>> significant re-architecture to make that possible. The two users of this
>> IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both
>> have only integrated the first source pad i.e stream0 of this IP. So for
>> now keep it simple and only allow 1-to-1 mapping of streams from sink to
>> source, without any broadcasting.
>>
>> With stream routing now supported in the driver, implement the
>> enable_stream and disable_stream hooks in place of the stream-unaware
>> s_stream hook.
>>
>> This allows consumer devices like a DMA bridge or ISP, to enable
>> particular streams on a source pad, which in turn can be used to enable
>> only particular streams on the CSI-TX device connected on the sink pad.
>>
>> Implement a fallback s_stream hook that internally calls enable_stream
>> on each source pad, for consumer drivers that don't use multi-stream
>> APIs to still work.
> Can't you use v4l2_subdev_s_stream_helper() ? If not, please briefly
> explain why in the commit message.
The v4l2_subdev_s_stream_helper() can only be used by subdevs that have
a single source pad, This IP is used by TI Shim and StarFive JH7110
CAMSS, currently both of them use only the first source pad (stream0)
but the driver can be re-architectured to use have more than one source
pad, this is the reason v4l2_subdev_s_stream_helper() is not used.
I will mention this in the commit message as well.
>
>> Signed-off-by: Jai Luthra <j-luthra@ti.com>
>> Co-developed-by: Rishikesh Donadkar <r-donadkar@ti.com>
>> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
>> ---
>> drivers/media/platform/cadence/cdns-csi2rx.c | 378 ++++++++++++++-----
>> 1 file changed, 288 insertions(+), 90 deletions(-)
>>
>> diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
>> index df05d278df379..95beb6623de8c 100644
>> --- a/drivers/media/platform/cadence/cdns-csi2rx.c
>> +++ b/drivers/media/platform/cadence/cdns-csi2rx.c
>> @@ -90,6 +90,7 @@ struct csi2rx_priv {
>> struct reset_control *pixel_rst[CSI2RX_STREAMS_MAX];
>> struct phy *dphy;
>>
>> + u32 vc_select[CSI2RX_STREAMS_MAX];
>> u8 lanes[CSI2RX_LANES_MAX];
>> u8 num_lanes;
>> u8 max_lanes;
>> @@ -179,29 +180,36 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
>>
>> static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
>> {
>> - struct media_pad *src_pad =
>> - &csi2rx->source_subdev->entity.pads[csi2rx->source_pad];
>> + struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
>> union phy_configure_opts opts = { };
>> struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
>> - struct v4l2_subdev_format sd_fmt = {
>> - .which = V4L2_SUBDEV_FORMAT_ACTIVE,
>> - .pad = CSI2RX_PAD_SINK,
>> - };
>> + struct v4l2_mbus_framefmt *framefmt;
>> + struct v4l2_subdev_state *state;
>> const struct csi2rx_fmt *fmt;
>> s64 link_freq;
>> int ret;
>>
>> - ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
>> - &sd_fmt);
>> - if (ret < 0)
>> - return ret;
>> -
>> - fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
>> + if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {
>> + link_freq = v4l2_get_link_freq(handler, 0, 0);
>> + } else {
>> + state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
>> + framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
>> + 0);
>> + fmt = csi2rx_get_fmt_by_code(framefmt->code);
>> +
>> + link_freq = v4l2_get_link_freq(handler, fmt->bpp,
>> + 2 * csi2rx->num_lanes);
>> +
>> + dev_warn(csi2rx->dev,
>> + "Guessing link frequency using bitdepth of stream 0.\n");
>> + dev_warn(csi2rx->dev,
>> + "V4L2_CID_LINK_FREQ control is required for multi format sources.\n");
>> + }
> Why do you need this complexity, instead of calling the
> v4l2_get_link_freq() variant that takes the source pad as its first
> argument ?
This was introduced to not break support for the single-stream
transmitters that do not implement V4L2_CID_LINK_FREQ. But now I realize
that v4l2_get_link_freq() checks if V4L2_CID_LINK_FREQ is present, if
not falls back to V4L2_CID_PIXEL_RATE for single-stream transmitters.
For multistream transmitters, V4L2_CID_LINK_FREQ should be implemented
as PIXEL_RATE does not make any sense, so I will call
v4l2_get_link_freq(handler, 0, 0) in multi-stream case.
This will force the multi-stream transmitter to implement
V4L2_CID_LINK_FREQ or else give an error.
>
>>
>> - link_freq = v4l2_get_link_freq(src_pad,
>> - fmt->bpp, 2 * csi2rx->num_lanes);
>> - if (link_freq < 0)
>> + if (link_freq < 0) {
>> + dev_err(csi2rx->dev, "Unable to calculate link frequency\n");
>> return link_freq;
>> + }
>>
>> ret = phy_mipi_dphy_get_default_config_for_hsclk(link_freq,
>> csi2rx->num_lanes, cfg);
>> @@ -224,18 +232,10 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
>> static int csi2rx_start(struct csi2rx_priv *csi2rx)
>> {
>> unsigned int i;
>> - struct media_pad *remote_pad;
>> unsigned long lanes_used = 0;
>> u32 reg;
>> int ret;
>>
>> - remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
>> - if (!remote_pad) {
>> - dev_err(csi2rx->dev,
>> - "Failed to find connected source\n");
>> - return -ENODEV;
>> - }
>> -
>> ret = clk_prepare_enable(csi2rx->p_clk);
>> if (ret)
>> return ret;
>> @@ -302,11 +302,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
>> writel(CSI2RX_STREAM_CFG_FIFO_MODE_LARGE_BUF,
>> csi2rx->base + CSI2RX_STREAM_CFG_REG(i));
>>
>> - /*
>> - * Enable one virtual channel. When multiple virtual channels
>> - * are supported this will have to be changed.
>> - */
>> - writel(CSI2RX_STREAM_DATA_CFG_VC_SELECT(0),
>> + writel(csi2rx->vc_select[i],
>> csi2rx->base + CSI2RX_STREAM_DATA_CFG_REG(i));
>>
>> writel(CSI2RX_STREAM_CTRL_START,
>> @@ -319,17 +315,10 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
>>
>> reset_control_deassert(csi2rx->sys_rst);
>>
>> - ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
>> - remote_pad->index, BIT(0));
>> - if (ret)
>> - goto err_disable_sysclk;
>> -
>> clk_disable_unprepare(csi2rx->p_clk);
>>
>> return 0;
>>
>> -err_disable_sysclk:
>> - clk_disable_unprepare(csi2rx->sys_clk);
>> err_disable_pixclk:
>> for (; i > 0; i--) {
>> reset_control_assert(csi2rx->pixel_rst[i - 1]);
>> @@ -348,7 +337,6 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
>>
>> static void csi2rx_stop(struct csi2rx_priv *csi2rx)
>> {
>> - struct media_pad *remote_pad;
>> unsigned int i;
>> u32 val;
>> int ret;
>> @@ -377,13 +365,6 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
>> reset_control_assert(csi2rx->p_rst);
>> clk_disable_unprepare(csi2rx->p_clk);
>>
>> - remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
>> - if (!remote_pad ||
>> - v4l2_subdev_disable_streams(csi2rx->source_subdev,
>> - remote_pad->index, BIT(0))) {
>> - dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
>> - }
>> -
>> if (csi2rx->dphy) {
>> writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
>>
>> @@ -392,37 +373,153 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
>> }
>> }
>>
>> -static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
>> +static void csi2rx_update_vc_select(struct csi2rx_priv *csi2rx,
>> + struct v4l2_subdev_state *state)
>> +{
>> + struct v4l2_mbus_frame_desc fd = {0};
>> + struct v4l2_subdev_route *route;
>> + unsigned int i;
>> + int ret;
>> +
>> + /* Capture VC=0 by default */
>> + for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
>> + csi2rx->vc_select[i] = CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);
>> +
>> + ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
>> + if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
>> + dev_dbg(csi2rx->dev,
>> + "Failed to get source frame desc, allowing only VC=0\n");
>> + return;
>> + }
>> +
>> + /* If source provides per-stream VC info, use it to filter by VC */
>> + memset(csi2rx->vc_select, 0, sizeof(csi2rx->vc_select));
>> +
>> + for_each_active_route(&state->routing, route) {
>> + u32 cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;
>> +
>> + for (i = 0; i < fd.num_entries; i++) {
>> + if (fd.entry[i].stream != route->sink_stream)
>> + continue;
>> +
>> + csi2rx->vc_select[cdns_stream] |=
>> + CSI2RX_STREAM_DATA_CFG_VC_SELECT(fd.entry[i].bus.csi2.vc);
>> + }
>> + }
>> +}
>> +
>> +static int csi2rx_enable_streams(struct v4l2_subdev *subdev,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask)
>> {
>> struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
>> - int ret = 0;
>> + struct media_pad *remote_pad;
>> + u64 sink_streams;
>> + int ret;
>> +
>> + remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
>> + if (!remote_pad) {
>> + dev_err(csi2rx->dev,
>> + "Failed to find connected source\n");
>> + return -ENODEV;
>> + }
>> +
>> + sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
>> + CSI2RX_PAD_SINK,
>> + &streams_mask);
>>
>> mutex_lock(&csi2rx->lock);
> Can you use lock guards ? It will simplify error handling.
Sure.
>
>> + /*
>> + * If we're not the first users, there's no need to
>> + * enable the whole controller.
>> + */
>> + if (!csi2rx->count) {
> I would have stored the bitmask of enabled streams instead of a count,
> but it doesn't matter that much.
>
>> + ret = csi2rx_start(csi2rx);
>> + if (ret)
>> + goto err_stream_start;
>> + }
>>
>> - if (enable) {
>> - /*
>> - * If we're not the first users, there's no need to
>> - * enable the whole controller.
>> - */
>> - if (!csi2rx->count) {
>> - ret = csi2rx_start(csi2rx);
>> - if (ret)
>> - goto out;
>> - }
>> + /* Start streaming on the source */
>> + ret = v4l2_subdev_enable_streams(csi2rx->source_subdev, remote_pad->index,
>> + sink_streams);
>> + if (ret) {
>> + dev_err(csi2rx->dev,
>> + "Failed to start streams %#llx on subdev\n",
>> + sink_streams);
>> + goto err_subdev_enable;
>> + }
>>
>> - csi2rx->count++;
>> - } else {
>> - csi2rx->count--;
>> + csi2rx->count++;
>> + mutex_unlock(&csi2rx->lock);
>> +
>> + return 0;
>> +
>> +err_subdev_enable:
>> + if (!csi2rx->count)
>> + csi2rx_stop(csi2rx);
>> +err_stream_start:
>> + mutex_unlock(&csi2rx->lock);
>> + return ret;
>> +}
>> +
>> +static int csi2rx_disable_streams(struct v4l2_subdev *subdev,
>> + struct v4l2_subdev_state *state, u32 pad,
>> + u64 streams_mask)
>> +{
>> + struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
>> + struct media_pad *remote_pad;
>> + u64 sink_streams;
>> +
>> + sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
>> + CSI2RX_PAD_SINK,
>> + &streams_mask);
>>
>> - /*
>> - * Let the last user turn off the lights.
>> - */
>> - if (!csi2rx->count)
>> - csi2rx_stop(csi2rx);
>> + remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
>> + if (!remote_pad ||
>> + v4l2_subdev_disable_streams(csi2rx->source_subdev,
>> + remote_pad->index, sink_streams)) {
>> + dev_err(csi2rx->dev, "Couldn't disable our subdev\n");
>> }
>>
>> -out:
>> + mutex_lock(&csi2rx->lock);
>> + csi2rx->count--;
>> + /*
>> + * Let the last user turn off the lights.
>> + */
> This can become a single line.
Will fix.
>
>> + if (!csi2rx->count)
>> + csi2rx_stop(csi2rx);
>> mutex_unlock(&csi2rx->lock);
>> +
>> + return 0;
>> +}
>> +
>> +static int csi2rx_s_stream_fallback(struct v4l2_subdev *sd, int enable)
>> +{
>> + struct v4l2_subdev_state *state;
>> + struct v4l2_subdev_route *route;
>> + u64 mask[CSI2RX_PAD_MAX] = {0};
>> + int i, ret;
>> +
>> + /* Find the stream mask on all source pads */
>> + state = v4l2_subdev_lock_and_get_active_state(sd);
>> + for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
>> + for_each_active_route(&state->routing, route) {
>> + if (route->source_pad == i)
>> + mask[i] |= BIT_ULL(route->source_stream);
>> + }
>> + }
>> + v4l2_subdev_unlock_state(state);
>> +
>> + /* Start streaming on each pad */
>> + for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
>> + if (enable)
>> + ret = v4l2_subdev_enable_streams(sd, i, mask[i]);
>> + else
>> + ret = v4l2_subdev_disable_streams(sd, i, mask[i]);
>> + if (ret)
>> + return ret;
>> + }
>> +
>> return ret;
>> }
>>
>> @@ -438,12 +535,58 @@ static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
>> return 0;
>> }
>>
>> +static int _csi2rx_set_routing(struct v4l2_subdev *subdev,
>> + struct v4l2_subdev_state *state,
>> + struct v4l2_subdev_krouting *routing)
>> +{
>> + static const struct v4l2_mbus_framefmt format = {
>> + .width = 640,
>> + .height = 480,
>> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
>> + .field = V4L2_FIELD_NONE,
>> + .colorspace = V4L2_COLORSPACE_SRGB,
>> + .ycbcr_enc = V4L2_YCBCR_ENC_601,
>> + .quantization = V4L2_QUANTIZATION_LIM_RANGE,
>> + .xfer_func = V4L2_XFER_FUNC_SRGB,
>> + };
>> + int ret;
>> +
>> + if (routing->num_routes > V4L2_FRAME_DESC_ENTRY_MAX)
>> + return -EINVAL;
>> +
>> + ret = v4l2_subdev_routing_validate(subdev, routing,
>> + V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
>> + if (ret)
>> + return ret;
>> +
>> + return v4l2_subdev_set_routing_with_fmt(subdev, state, routing, &format);
>> +}
>> +
>> +static int csi2rx_set_routing(struct v4l2_subdev *subdev,
>> + struct v4l2_subdev_state *state,
>> + enum v4l2_subdev_format_whence which,
>> + struct v4l2_subdev_krouting *routing)
>> +{
>> + struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
>> + int ret;
>> +
>> + if (which == V4L2_SUBDEV_FORMAT_ACTIVE && csi2rx->count)
>> + return -EBUSY;
>> +
>> + ret = _csi2rx_set_routing(subdev, state, routing);
>> + if (ret)
>> + return ret;
>> +
>> + csi2rx_update_vc_select(csi2rx, state);
> When setting routing on the TRY state this will update the active state,
> which isn't correct. Can you call this function when starting the device
> instead ?
Yes, this can be called at stream start, before configuring the VC data
in the registers.
Will do this in the next revision.
>
>> +
>> + return 0;
>> +}
>> +
>> static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
>> struct v4l2_subdev_state *state,
>> struct v4l2_subdev_format *format)
>> {
>> struct v4l2_mbus_framefmt *fmt;
>> - unsigned int i;
>>
>> /* No transcoding, source and sink formats must match. */
>> if (format->pad != CSI2RX_PAD_SINK)
>> @@ -455,14 +598,16 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
>> format->format.field = V4L2_FIELD_NONE;
>>
>> /* Set sink format */
>> - fmt = v4l2_subdev_state_get_format(state, format->pad);
>> + fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
>> *fmt = format->format;
>>
>> - /* Propagate to source formats */
>> - for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
>> - fmt = v4l2_subdev_state_get_format(state, i);
>> - *fmt = format->format;
>> - }
>> + /* Propagate to source format */
>> + fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
>> + format->stream);
>> + if (!fmt)
>> + return -EINVAL;
>> +
>> + *fmt = format->format;
>>
>> return 0;
>> }
>> @@ -470,40 +615,92 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
>> static int csi2rx_init_state(struct v4l2_subdev *subdev,
>> struct v4l2_subdev_state *state)
>> {
>> - struct v4l2_subdev_format format = {
>> - .pad = CSI2RX_PAD_SINK,
>> - .format = {
>> - .width = 640,
>> - .height = 480,
>> - .code = MEDIA_BUS_FMT_UYVY8_1X16,
>> - .field = V4L2_FIELD_NONE,
>> - .colorspace = V4L2_COLORSPACE_SRGB,
>> - .ycbcr_enc = V4L2_YCBCR_ENC_601,
>> - .quantization = V4L2_QUANTIZATION_LIM_RANGE,
>> - .xfer_func = V4L2_XFER_FUNC_SRGB,
>> + struct v4l2_subdev_route routes[] = {
>> + {
>> + .sink_pad = CSI2RX_PAD_SINK,
>> + .sink_stream = 0,
>> + .source_pad = CSI2RX_PAD_SOURCE_STREAM0,
>> + .source_stream = 0,
>> + .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
>> },
>> };
>>
>> - return csi2rx_set_fmt(subdev, state, &format);
>> + struct v4l2_subdev_krouting routing = {
>> + .num_routes = ARRAY_SIZE(routes),
>> + .routes = routes,
>> + };
>> +
>> + return _csi2rx_set_routing(subdev, state, &routing);
>> }
>>
>> static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
>> struct v4l2_mbus_frame_desc *fd)
>> {
>> struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
>> + struct v4l2_mbus_frame_desc source_fd = {0};
>> + struct v4l2_subdev_route *route;
>> + struct v4l2_subdev_state *state;
>> + int ret;
>>
>> - return csi2rx_get_frame_desc_from_source(csi2rx, fd);
>> + ret = csi2rx_get_frame_desc_from_source(csi2rx, &source_fd);
>> + if (ret)
>> + return ret;
>> +
>> + fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
>> +
>> + state = v4l2_subdev_lock_and_get_active_state(subdev);
>> +
>> + for_each_active_route(&state->routing, route) {
>> + struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
>> + unsigned int i;
>> +
>> + if (route->source_pad != pad)
>> + continue;
>> +
>> + for (i = 0; i < source_fd.num_entries; i++) {
>> + if (source_fd.entry[i].stream == route->sink_stream) {
>> + source_entry = &source_fd.entry[i];
>> + break;
>> + }
>> + }
>> +
>> + if (!source_entry) {
>> + dev_err(csi2rx->dev,
>> + "Failed to find stream from source frame desc\n");
>> + ret = -EPIPE;
>> + goto err_missing_stream;
>> + }
>> +
>> + fd->entry[fd->num_entries].stream = route->source_stream;
>> + fd->entry[fd->num_entries].flags = source_entry->flags;
>> + fd->entry[fd->num_entries].length = source_entry->length;
>> + fd->entry[fd->num_entries].pixelcode = source_entry->pixelcode;
>> + fd->entry[fd->num_entries].bus.csi2.vc =
>> + source_entry->bus.csi2.vc;
>> + fd->entry[fd->num_entries].bus.csi2.dt =
>> + source_entry->bus.csi2.dt;
>> +
>> + fd->num_entries++;
>> + }
>> +
>> +err_missing_stream:
>> + v4l2_subdev_unlock_state(state);
>> +
>> + return ret;
>> }
>>
>> static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
>> - .enum_mbus_code = csi2rx_enum_mbus_code,
>> - .get_fmt = v4l2_subdev_get_fmt,
>> - .set_fmt = csi2rx_set_fmt,
>> - .get_frame_desc = csi2rx_get_frame_desc,
>> + .enum_mbus_code = csi2rx_enum_mbus_code,
>> + .get_fmt = v4l2_subdev_get_fmt,
>> + .set_fmt = csi2rx_set_fmt,
>> + .get_frame_desc = csi2rx_get_frame_desc,
>> + .set_routing = csi2rx_set_routing,
>> + .enable_streams = csi2rx_enable_streams,
>> + .disable_streams = csi2rx_disable_streams,
>> };
>>
>> static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
>> - .s_stream = csi2rx_s_stream,
>> + .s_stream = csi2rx_s_stream_fallback,
>> };
>>
>> static const struct v4l2_subdev_ops csi2rx_subdev_ops = {
>> @@ -735,7 +932,8 @@ static int csi2rx_probe(struct platform_device *pdev)
>> csi2rx->pads[CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++)
>> csi2rx->pads[i].flags = MEDIA_PAD_FL_SOURCE;
>> - csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>> + csi2rx->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
>> + V4L2_SUBDEV_FL_STREAMS;
>> csi2rx->subdev.entity.ops = &csi2rx_media_ops;
>>
>> ret = media_entity_pads_init(&csi2rx->subdev.entity, CSI2RX_PAD_MAX,
Regards,
Rishikesh
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support
2025-05-06 11:06 ` Rishikesh Donadkar
@ 2025-05-09 8:47 ` Laurent Pinchart
0 siblings, 0 replies; 36+ messages in thread
From: Laurent Pinchart @ 2025-05-09 8:47 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: jai.luthra, mripard, linux-kernel, linux-media, devicetree,
devarsht, y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt,
vaishnav.a, s-jain1, vigneshr, sakari.ailus, hverkuil-cisco,
tomi.valkeinen, jai.luthra, changhuang.liang, jack.zhu
Hi Rishikesh,
On Tue, May 06, 2025 at 04:36:58PM +0530, Rishikesh Donadkar wrote:
> On 21/04/25 19:27, Laurent Pinchart wrote:
> > On Thu, Apr 17, 2025 at 12:25:51PM +0530, Rishikesh Donadkar wrote:
> >> From: Jai Luthra <j-luthra@ti.com>
> >>
> >> Cadence CSI-2 bridge IP supports capturing multiple virtual "streams"
> >> of data over the same physical interface using MIPI Virtual Channels.
> >>
> >> The V4L2 subdev APIs should reflect this capability and allow per-stream
> >> routing and controls.
> >>
> >> While the hardware IP supports usecases where streams coming in the sink
> >> pad can be broadcasted to multiple source pads, the driver will need
> >> significant re-architecture to make that possible. The two users of this
> >> IP in mainline linux are TI Shim and StarFive JH7110 CAMSS, and both
> >> have only integrated the first source pad i.e stream0 of this IP. So for
> >> now keep it simple and only allow 1-to-1 mapping of streams from sink to
> >> source, without any broadcasting.
> >>
> >> With stream routing now supported in the driver, implement the
> >> enable_stream and disable_stream hooks in place of the stream-unaware
> >> s_stream hook.
> >>
> >> This allows consumer devices like a DMA bridge or ISP, to enable
> >> particular streams on a source pad, which in turn can be used to enable
> >> only particular streams on the CSI-TX device connected on the sink pad.
> >>
> >> Implement a fallback s_stream hook that internally calls enable_stream
> >> on each source pad, for consumer drivers that don't use multi-stream
> >> APIs to still work.
> >
> > Can't you use v4l2_subdev_s_stream_helper() ? If not, please briefly
> > explain why in the commit message.
>
> The v4l2_subdev_s_stream_helper() can only be used by subdevs that have
> a single source pad, This IP is used by TI Shim and StarFive JH7110
> CAMSS, currently both of them use only the first source pad (stream0)
> but the driver can be re-architectured to use have more than one source
> pad, this is the reason v4l2_subdev_s_stream_helper() is not used.
>
> I will mention this in the commit message as well.
>
> >> Signed-off-by: Jai Luthra <j-luthra@ti.com>
> >> Co-developed-by: Rishikesh Donadkar <r-donadkar@ti.com>
> >> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> >> ---
> >> drivers/media/platform/cadence/cdns-csi2rx.c | 378 ++++++++++++++-----
> >> 1 file changed, 288 insertions(+), 90 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
> >> index df05d278df379..95beb6623de8c 100644
> >> --- a/drivers/media/platform/cadence/cdns-csi2rx.c
> >> +++ b/drivers/media/platform/cadence/cdns-csi2rx.c
> >> @@ -90,6 +90,7 @@ struct csi2rx_priv {
> >> struct reset_control *pixel_rst[CSI2RX_STREAMS_MAX];
> >> struct phy *dphy;
> >>
> >> + u32 vc_select[CSI2RX_STREAMS_MAX];
> >> u8 lanes[CSI2RX_LANES_MAX];
> >> u8 num_lanes;
> >> u8 max_lanes;
> >> @@ -179,29 +180,36 @@ static void csi2rx_reset(struct csi2rx_priv *csi2rx)
> >>
> >> static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
> >> {
> >> - struct media_pad *src_pad =
> >> - &csi2rx->source_subdev->entity.pads[csi2rx->source_pad];
> >> + struct v4l2_ctrl_handler *handler = csi2rx->source_subdev->ctrl_handler;
> >> union phy_configure_opts opts = { };
> >> struct phy_configure_opts_mipi_dphy *cfg = &opts.mipi_dphy;
> >> - struct v4l2_subdev_format sd_fmt = {
> >> - .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> >> - .pad = CSI2RX_PAD_SINK,
> >> - };
> >> + struct v4l2_mbus_framefmt *framefmt;
> >> + struct v4l2_subdev_state *state;
> >> const struct csi2rx_fmt *fmt;
> >> s64 link_freq;
> >> int ret;
> >>
> >> - ret = v4l2_subdev_call_state_active(&csi2rx->subdev, pad, get_fmt,
> >> - &sd_fmt);
> >> - if (ret < 0)
> >> - return ret;
> >> -
> >> - fmt = csi2rx_get_fmt_by_code(sd_fmt.format.code);
> >> + if (v4l2_ctrl_find(handler, V4L2_CID_LINK_FREQ)) {
> >> + link_freq = v4l2_get_link_freq(handler, 0, 0);
> >> + } else {
> >> + state = v4l2_subdev_get_locked_active_state(&csi2rx->subdev);
> >> + framefmt = v4l2_subdev_state_get_format(state, CSI2RX_PAD_SINK,
> >> + 0);
> >> + fmt = csi2rx_get_fmt_by_code(framefmt->code);
> >> +
> >> + link_freq = v4l2_get_link_freq(handler, fmt->bpp,
> >> + 2 * csi2rx->num_lanes);
> >> +
> >> + dev_warn(csi2rx->dev,
> >> + "Guessing link frequency using bitdepth of stream 0.\n");
> >> + dev_warn(csi2rx->dev,
> >> + "V4L2_CID_LINK_FREQ control is required for multi format sources.\n");
> >> + }
> >
> > Why do you need this complexity, instead of calling the
> > v4l2_get_link_freq() variant that takes the source pad as its first
> > argument ?
>
> This was introduced to not break support for the single-stream
> transmitters that do not implement V4L2_CID_LINK_FREQ. But now I realize
> that v4l2_get_link_freq() checks if V4L2_CID_LINK_FREQ is present, if
> not falls back to V4L2_CID_PIXEL_RATE for single-stream transmitters.
>
> For multistream transmitters, V4L2_CID_LINK_FREQ should be implemented
> as PIXEL_RATE does not make any sense, so I will call
> v4l2_get_link_freq(handler, 0, 0) in multi-stream case.
>
> This will force the multi-stream transmitter to implement
> V4L2_CID_LINK_FREQ or else give an error.
I think that's fine, we want drivers to move towards the recommended
APIs.
> >>
> >> - link_freq = v4l2_get_link_freq(src_pad,
> >> - fmt->bpp, 2 * csi2rx->num_lanes);
> >> - if (link_freq < 0)
> >> + if (link_freq < 0) {
> >> + dev_err(csi2rx->dev, "Unable to calculate link frequency\n");
> >> return link_freq;
> >> + }
> >>
> >> ret = phy_mipi_dphy_get_default_config_for_hsclk(link_freq,
> >> csi2rx->num_lanes, cfg);
> >> @@ -224,18 +232,10 @@ static int csi2rx_configure_ext_dphy(struct csi2rx_priv *csi2rx)
> >> static int csi2rx_start(struct csi2rx_priv *csi2rx)
> >> {
> >> unsigned int i;
> >> - struct media_pad *remote_pad;
> >> unsigned long lanes_used = 0;
> >> u32 reg;
> >> int ret;
> >>
> >> - remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> >> - if (!remote_pad) {
> >> - dev_err(csi2rx->dev,
> >> - "Failed to find connected source\n");
> >> - return -ENODEV;
> >> - }
> >> -
> >> ret = clk_prepare_enable(csi2rx->p_clk);
> >> if (ret)
> >> return ret;
> >> @@ -302,11 +302,7 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
> >> writel(CSI2RX_STREAM_CFG_FIFO_MODE_LARGE_BUF,
> >> csi2rx->base + CSI2RX_STREAM_CFG_REG(i));
> >>
> >> - /*
> >> - * Enable one virtual channel. When multiple virtual channels
> >> - * are supported this will have to be changed.
> >> - */
> >> - writel(CSI2RX_STREAM_DATA_CFG_VC_SELECT(0),
> >> + writel(csi2rx->vc_select[i],
> >> csi2rx->base + CSI2RX_STREAM_DATA_CFG_REG(i));
> >>
> >> writel(CSI2RX_STREAM_CTRL_START,
> >> @@ -319,17 +315,10 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
> >>
> >> reset_control_deassert(csi2rx->sys_rst);
> >>
> >> - ret = v4l2_subdev_enable_streams(csi2rx->source_subdev,
> >> - remote_pad->index, BIT(0));
> >> - if (ret)
> >> - goto err_disable_sysclk;
> >> -
> >> clk_disable_unprepare(csi2rx->p_clk);
> >>
> >> return 0;
> >>
> >> -err_disable_sysclk:
> >> - clk_disable_unprepare(csi2rx->sys_clk);
> >> err_disable_pixclk:
> >> for (; i > 0; i--) {
> >> reset_control_assert(csi2rx->pixel_rst[i - 1]);
> >> @@ -348,7 +337,6 @@ static int csi2rx_start(struct csi2rx_priv *csi2rx)
> >>
> >> static void csi2rx_stop(struct csi2rx_priv *csi2rx)
> >> {
> >> - struct media_pad *remote_pad;
> >> unsigned int i;
> >> u32 val;
> >> int ret;
> >> @@ -377,13 +365,6 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
> >> reset_control_assert(csi2rx->p_rst);
> >> clk_disable_unprepare(csi2rx->p_clk);
> >>
> >> - remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> >> - if (!remote_pad ||
> >> - v4l2_subdev_disable_streams(csi2rx->source_subdev,
> >> - remote_pad->index, BIT(0))) {
> >> - dev_warn(csi2rx->dev, "Couldn't disable our subdev\n");
> >> - }
> >> -
> >> if (csi2rx->dphy) {
> >> writel(0, csi2rx->base + CSI2RX_DPHY_LANE_CTRL_REG);
> >>
> >> @@ -392,37 +373,153 @@ static void csi2rx_stop(struct csi2rx_priv *csi2rx)
> >> }
> >> }
> >>
> >> -static int csi2rx_s_stream(struct v4l2_subdev *subdev, int enable)
> >> +static void csi2rx_update_vc_select(struct csi2rx_priv *csi2rx,
> >> + struct v4l2_subdev_state *state)
> >> +{
> >> + struct v4l2_mbus_frame_desc fd = {0};
> >> + struct v4l2_subdev_route *route;
> >> + unsigned int i;
> >> + int ret;
> >> +
> >> + /* Capture VC=0 by default */
> >> + for (i = 0; i < CSI2RX_STREAMS_MAX; i++)
> >> + csi2rx->vc_select[i] = CSI2RX_STREAM_DATA_CFG_VC_SELECT(0);
> >> +
> >> + ret = csi2rx_get_frame_desc_from_source(csi2rx, &fd);
> >> + if (ret || fd.type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
> >> + dev_dbg(csi2rx->dev,
> >> + "Failed to get source frame desc, allowing only VC=0\n");
> >> + return;
> >> + }
> >> +
> >> + /* If source provides per-stream VC info, use it to filter by VC */
> >> + memset(csi2rx->vc_select, 0, sizeof(csi2rx->vc_select));
> >> +
> >> + for_each_active_route(&state->routing, route) {
> >> + u32 cdns_stream = route->source_pad - CSI2RX_PAD_SOURCE_STREAM0;
> >> +
> >> + for (i = 0; i < fd.num_entries; i++) {
> >> + if (fd.entry[i].stream != route->sink_stream)
> >> + continue;
> >> +
> >> + csi2rx->vc_select[cdns_stream] |=
> >> + CSI2RX_STREAM_DATA_CFG_VC_SELECT(fd.entry[i].bus.csi2.vc);
> >> + }
> >> + }
> >> +}
> >> +
> >> +static int csi2rx_enable_streams(struct v4l2_subdev *subdev,
> >> + struct v4l2_subdev_state *state, u32 pad,
> >> + u64 streams_mask)
> >> {
> >> struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> >> - int ret = 0;
> >> + struct media_pad *remote_pad;
> >> + u64 sink_streams;
> >> + int ret;
> >> +
> >> + remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> >> + if (!remote_pad) {
> >> + dev_err(csi2rx->dev,
> >> + "Failed to find connected source\n");
> >> + return -ENODEV;
> >> + }
> >> +
> >> + sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
> >> + CSI2RX_PAD_SINK,
> >> + &streams_mask);
> >>
> >> mutex_lock(&csi2rx->lock);
> >
> > Can you use lock guards ? It will simplify error handling.
>
> Sure.
>
> >> + /*
> >> + * If we're not the first users, there's no need to
> >> + * enable the whole controller.
> >> + */
> >> + if (!csi2rx->count) {
> >
> > I would have stored the bitmask of enabled streams instead of a count,
> > but it doesn't matter that much.
> >
> >> + ret = csi2rx_start(csi2rx);
> >> + if (ret)
> >> + goto err_stream_start;
> >> + }
> >>
> >> - if (enable) {
> >> - /*
> >> - * If we're not the first users, there's no need to
> >> - * enable the whole controller.
> >> - */
> >> - if (!csi2rx->count) {
> >> - ret = csi2rx_start(csi2rx);
> >> - if (ret)
> >> - goto out;
> >> - }
> >> + /* Start streaming on the source */
> >> + ret = v4l2_subdev_enable_streams(csi2rx->source_subdev, remote_pad->index,
> >> + sink_streams);
> >> + if (ret) {
> >> + dev_err(csi2rx->dev,
> >> + "Failed to start streams %#llx on subdev\n",
> >> + sink_streams);
> >> + goto err_subdev_enable;
> >> + }
> >>
> >> - csi2rx->count++;
> >> - } else {
> >> - csi2rx->count--;
> >> + csi2rx->count++;
> >> + mutex_unlock(&csi2rx->lock);
> >> +
> >> + return 0;
> >> +
> >> +err_subdev_enable:
> >> + if (!csi2rx->count)
> >> + csi2rx_stop(csi2rx);
> >> +err_stream_start:
> >> + mutex_unlock(&csi2rx->lock);
> >> + return ret;
> >> +}
> >> +
> >> +static int csi2rx_disable_streams(struct v4l2_subdev *subdev,
> >> + struct v4l2_subdev_state *state, u32 pad,
> >> + u64 streams_mask)
> >> +{
> >> + struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> >> + struct media_pad *remote_pad;
> >> + u64 sink_streams;
> >> +
> >> + sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
> >> + CSI2RX_PAD_SINK,
> >> + &streams_mask);
> >>
> >> - /*
> >> - * Let the last user turn off the lights.
> >> - */
> >> - if (!csi2rx->count)
> >> - csi2rx_stop(csi2rx);
> >> + remote_pad = media_pad_remote_pad_first(&csi2rx->pads[CSI2RX_PAD_SINK]);
> >> + if (!remote_pad ||
> >> + v4l2_subdev_disable_streams(csi2rx->source_subdev,
> >> + remote_pad->index, sink_streams)) {
> >> + dev_err(csi2rx->dev, "Couldn't disable our subdev\n");
> >> }
> >>
> >> -out:
> >> + mutex_lock(&csi2rx->lock);
> >> + csi2rx->count--;
> >> + /*
> >> + * Let the last user turn off the lights.
> >> + */
> >
> > This can become a single line.
>
> Will fix.
>
> >> + if (!csi2rx->count)
> >> + csi2rx_stop(csi2rx);
> >> mutex_unlock(&csi2rx->lock);
> >> +
> >> + return 0;
> >> +}
> >> +
> >> +static int csi2rx_s_stream_fallback(struct v4l2_subdev *sd, int enable)
> >> +{
> >> + struct v4l2_subdev_state *state;
> >> + struct v4l2_subdev_route *route;
> >> + u64 mask[CSI2RX_PAD_MAX] = {0};
> >> + int i, ret;
> >> +
> >> + /* Find the stream mask on all source pads */
> >> + state = v4l2_subdev_lock_and_get_active_state(sd);
> >> + for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
> >> + for_each_active_route(&state->routing, route) {
> >> + if (route->source_pad == i)
> >> + mask[i] |= BIT_ULL(route->source_stream);
> >> + }
> >> + }
> >> + v4l2_subdev_unlock_state(state);
> >> +
> >> + /* Start streaming on each pad */
> >> + for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
> >> + if (enable)
> >> + ret = v4l2_subdev_enable_streams(sd, i, mask[i]);
> >> + else
> >> + ret = v4l2_subdev_disable_streams(sd, i, mask[i]);
> >> + if (ret)
> >> + return ret;
> >> + }
> >> +
> >> return ret;
> >> }
> >>
> >> @@ -438,12 +535,58 @@ static int csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
> >> return 0;
> >> }
> >>
> >> +static int _csi2rx_set_routing(struct v4l2_subdev *subdev,
> >> + struct v4l2_subdev_state *state,
> >> + struct v4l2_subdev_krouting *routing)
> >> +{
> >> + static const struct v4l2_mbus_framefmt format = {
> >> + .width = 640,
> >> + .height = 480,
> >> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
> >> + .field = V4L2_FIELD_NONE,
> >> + .colorspace = V4L2_COLORSPACE_SRGB,
> >> + .ycbcr_enc = V4L2_YCBCR_ENC_601,
> >> + .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> >> + .xfer_func = V4L2_XFER_FUNC_SRGB,
> >> + };
> >> + int ret;
> >> +
> >> + if (routing->num_routes > V4L2_FRAME_DESC_ENTRY_MAX)
> >> + return -EINVAL;
> >> +
> >> + ret = v4l2_subdev_routing_validate(subdev, routing,
> >> + V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
> >> + if (ret)
> >> + return ret;
> >> +
> >> + return v4l2_subdev_set_routing_with_fmt(subdev, state, routing, &format);
> >> +}
> >> +
> >> +static int csi2rx_set_routing(struct v4l2_subdev *subdev,
> >> + struct v4l2_subdev_state *state,
> >> + enum v4l2_subdev_format_whence which,
> >> + struct v4l2_subdev_krouting *routing)
> >> +{
> >> + struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> >> + int ret;
> >> +
> >> + if (which == V4L2_SUBDEV_FORMAT_ACTIVE && csi2rx->count)
> >> + return -EBUSY;
> >> +
> >> + ret = _csi2rx_set_routing(subdev, state, routing);
> >> + if (ret)
> >> + return ret;
> >> +
> >> + csi2rx_update_vc_select(csi2rx, state);
> >
> > When setting routing on the TRY state this will update the active state,
> > which isn't correct. Can you call this function when starting the device
> > instead ?
>
> Yes, this can be called at stream start, before configuring the VC data
> in the registers.
>
> Will do this in the next revision.
>
> >> +
> >> + return 0;
> >> +}
> >> +
> >> static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
> >> struct v4l2_subdev_state *state,
> >> struct v4l2_subdev_format *format)
> >> {
> >> struct v4l2_mbus_framefmt *fmt;
> >> - unsigned int i;
> >>
> >> /* No transcoding, source and sink formats must match. */
> >> if (format->pad != CSI2RX_PAD_SINK)
> >> @@ -455,14 +598,16 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
> >> format->format.field = V4L2_FIELD_NONE;
> >>
> >> /* Set sink format */
> >> - fmt = v4l2_subdev_state_get_format(state, format->pad);
> >> + fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
> >> *fmt = format->format;
> >>
> >> - /* Propagate to source formats */
> >> - for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++) {
> >> - fmt = v4l2_subdev_state_get_format(state, i);
> >> - *fmt = format->format;
> >> - }
> >> + /* Propagate to source format */
> >> + fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
> >> + format->stream);
> >> + if (!fmt)
> >> + return -EINVAL;
> >> +
> >> + *fmt = format->format;
> >>
> >> return 0;
> >> }
> >> @@ -470,40 +615,92 @@ static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
> >> static int csi2rx_init_state(struct v4l2_subdev *subdev,
> >> struct v4l2_subdev_state *state)
> >> {
> >> - struct v4l2_subdev_format format = {
> >> - .pad = CSI2RX_PAD_SINK,
> >> - .format = {
> >> - .width = 640,
> >> - .height = 480,
> >> - .code = MEDIA_BUS_FMT_UYVY8_1X16,
> >> - .field = V4L2_FIELD_NONE,
> >> - .colorspace = V4L2_COLORSPACE_SRGB,
> >> - .ycbcr_enc = V4L2_YCBCR_ENC_601,
> >> - .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> >> - .xfer_func = V4L2_XFER_FUNC_SRGB,
> >> + struct v4l2_subdev_route routes[] = {
> >> + {
> >> + .sink_pad = CSI2RX_PAD_SINK,
> >> + .sink_stream = 0,
> >> + .source_pad = CSI2RX_PAD_SOURCE_STREAM0,
> >> + .source_stream = 0,
> >> + .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> >> },
> >> };
> >>
> >> - return csi2rx_set_fmt(subdev, state, &format);
> >> + struct v4l2_subdev_krouting routing = {
> >> + .num_routes = ARRAY_SIZE(routes),
> >> + .routes = routes,
> >> + };
> >> +
> >> + return _csi2rx_set_routing(subdev, state, &routing);
> >> }
> >>
> >> static int csi2rx_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
> >> struct v4l2_mbus_frame_desc *fd)
> >> {
> >> struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
> >> + struct v4l2_mbus_frame_desc source_fd = {0};
> >> + struct v4l2_subdev_route *route;
> >> + struct v4l2_subdev_state *state;
> >> + int ret;
> >>
> >> - return csi2rx_get_frame_desc_from_source(csi2rx, fd);
> >> + ret = csi2rx_get_frame_desc_from_source(csi2rx, &source_fd);
> >> + if (ret)
> >> + return ret;
> >> +
> >> + fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
> >> +
> >> + state = v4l2_subdev_lock_and_get_active_state(subdev);
> >> +
> >> + for_each_active_route(&state->routing, route) {
> >> + struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
> >> + unsigned int i;
> >> +
> >> + if (route->source_pad != pad)
> >> + continue;
> >> +
> >> + for (i = 0; i < source_fd.num_entries; i++) {
> >> + if (source_fd.entry[i].stream == route->sink_stream) {
> >> + source_entry = &source_fd.entry[i];
> >> + break;
> >> + }
> >> + }
> >> +
> >> + if (!source_entry) {
> >> + dev_err(csi2rx->dev,
> >> + "Failed to find stream from source frame desc\n");
> >> + ret = -EPIPE;
> >> + goto err_missing_stream;
> >> + }
> >> +
> >> + fd->entry[fd->num_entries].stream = route->source_stream;
> >> + fd->entry[fd->num_entries].flags = source_entry->flags;
> >> + fd->entry[fd->num_entries].length = source_entry->length;
> >> + fd->entry[fd->num_entries].pixelcode = source_entry->pixelcode;
> >> + fd->entry[fd->num_entries].bus.csi2.vc =
> >> + source_entry->bus.csi2.vc;
> >> + fd->entry[fd->num_entries].bus.csi2.dt =
> >> + source_entry->bus.csi2.dt;
> >> +
> >> + fd->num_entries++;
> >> + }
> >> +
> >> +err_missing_stream:
> >> + v4l2_subdev_unlock_state(state);
> >> +
> >> + return ret;
> >> }
> >>
> >> static const struct v4l2_subdev_pad_ops csi2rx_pad_ops = {
> >> - .enum_mbus_code = csi2rx_enum_mbus_code,
> >> - .get_fmt = v4l2_subdev_get_fmt,
> >> - .set_fmt = csi2rx_set_fmt,
> >> - .get_frame_desc = csi2rx_get_frame_desc,
> >> + .enum_mbus_code = csi2rx_enum_mbus_code,
> >> + .get_fmt = v4l2_subdev_get_fmt,
> >> + .set_fmt = csi2rx_set_fmt,
> >> + .get_frame_desc = csi2rx_get_frame_desc,
> >> + .set_routing = csi2rx_set_routing,
> >> + .enable_streams = csi2rx_enable_streams,
> >> + .disable_streams = csi2rx_disable_streams,
> >> };
> >>
> >> static const struct v4l2_subdev_video_ops csi2rx_video_ops = {
> >> - .s_stream = csi2rx_s_stream,
> >> + .s_stream = csi2rx_s_stream_fallback,
> >> };
> >>
> >> static const struct v4l2_subdev_ops csi2rx_subdev_ops = {
> >> @@ -735,7 +932,8 @@ static int csi2rx_probe(struct platform_device *pdev)
> >> csi2rx->pads[CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> >> for (i = CSI2RX_PAD_SOURCE_STREAM0; i < CSI2RX_PAD_MAX; i++)
> >> csi2rx->pads[i].flags = MEDIA_PAD_FL_SOURCE;
> >> - csi2rx->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> >> + csi2rx->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
> >> + V4L2_SUBDEV_FL_STREAMS;
> >> csi2rx->subdev.entity.ops = &csi2rx_media_ops;
> >>
> >> ret = media_entity_pads_init(&csi2rx->subdev.entity, CSI2RX_PAD_MAX,
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 36+ messages in thread
* [PATCH v3 11/13] media: ti: j721e-csi2rx: add multistream support
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (9 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 10/13] media: cadence: csi2rx: Enable multi-stream support Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-17 6:55 ` [PATCH v3 12/13] media: ti: j721e-csi2rx: Submit all available buffers Rishikesh Donadkar
2025-04-17 6:55 ` [PATCH v3 13/13] media: ti: j721e-csi2rx: Change the drain architecture for multistream Rishikesh Donadkar
12 siblings, 0 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Jai Luthra <j-luthra@ti.com>
Each CSI2 stream can be multiplexed into 4 independent streams, each
identified by its virtual channel number. To capture this multiplexed
stream, the application needs to tell the driver how it wants to route
the data. It needs to specify which context should process which stream.
This is done via the new routing APIs.
Add ioctls to accept routing information from the application and save
that in the driver. This can be used when starting streaming on a
context to determine which route and consequently which virtual channel
it should process.
Support the new enable_stream()/disable_stream() APIs in the subdev
instead of s_stream() hook.
De-assert the pixel interface reset on first start_streaming() and assert
it on the last stop_streaming().
Co-developed-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Pratyush Yadav <p.yadav@ti.com>
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Co-developed-by: Rishikesh Donadkar <r-donadkar@ti.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 212 ++++++++++++++----
1 file changed, 166 insertions(+), 46 deletions(-)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index 3e2a0517a9096..d00b5a1848043 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -538,8 +538,10 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
fmt = find_format_by_fourcc(ctx->v_fmt.fmt.pix.pixelformat);
/* De-assert the pixel interface reset. */
- reg = SHIM_CNTL_PIX_RST;
- writel(reg, csi->shim + SHIM_CNTL);
+ if (!csi->enable_count) {
+ reg = SHIM_CNTL_PIX_RST;
+ writel(reg, csi->shim + SHIM_CNTL);
+ }
reg = SHIM_DMACNTX_EN;
reg |= FIELD_PREP(SHIM_DMACNTX_FMT, fmt->csi_dt);
@@ -883,8 +885,12 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
struct ti_csi2rx_dev *csi = ctx->csi;
struct ti_csi2rx_dma *dma = &ctx->dma;
struct ti_csi2rx_buffer *buf;
+ struct v4l2_subdev_krouting *routing;
+ struct v4l2_subdev_route *route = NULL, *r;
+ struct media_pad *remote_pad;
unsigned long flags;
int ret = 0;
+ struct v4l2_subdev_state *state;
spin_lock_irqsave(&dma->lock, flags);
if (list_empty(&dma->queue))
@@ -897,6 +903,38 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
if (ret)
goto err;
+ remote_pad = media_entity_remote_source_pad_unique(ctx->pad.entity);
+ if (!remote_pad) {
+ ret = -ENODEV;
+ goto err;
+ }
+
+ state = v4l2_subdev_lock_and_get_active_state(&csi->subdev);
+
+ routing = &state->routing;
+
+ /* Find the stream to process. */
+ for_each_active_route(routing, r) {
+ if (!(r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+ continue;
+
+ if (r->source_pad != remote_pad->index)
+ continue;
+
+ route = r;
+ break;
+ }
+
+ if (!route) {
+ ret = -ENODEV;
+ v4l2_subdev_unlock_state(state);
+ goto err;
+ }
+
+ ctx->stream = route->sink_stream;
+
+ v4l2_subdev_unlock_state(state);
+
ret = ti_csi2rx_get_vc(ctx);
if (ret == -ENOIOCTLCMD)
ctx->vc = 0;
@@ -923,7 +961,10 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
dma->state = TI_CSI2RX_DMA_ACTIVE;
spin_unlock_irqrestore(&dma->lock, flags);
- ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 1);
+ /* Start stream 0, we don't allow multiple streams on the source pad */
+ ret = v4l2_subdev_enable_streams(&csi->subdev,
+ TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+ BIT(0));
if (ret)
goto err_dma;
@@ -946,12 +987,16 @@ static void ti_csi2rx_stop_streaming(struct vb2_queue *vq)
struct ti_csi2rx_dev *csi = ctx->csi;
int ret;
+ /* assert pixel reset to prevent stale data */
+ if (csi->enable_count == 1)
+ writel(0, csi->shim + SHIM_CNTL);
+
video_device_pipeline_stop(&ctx->vdev);
- writel(0, csi->shim + SHIM_CNTL);
- writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
+ ret = v4l2_subdev_disable_streams(&csi->subdev,
+ TI_CSI2RX_PAD_FIRST_SOURCE + ctx->idx,
+ BIT(0));
- ret = v4l2_subdev_call(&csi->subdev, video, s_stream, 0);
if (ret)
dev_err(csi->dev, "Failed to stop subdev stream\n");
@@ -990,79 +1035,154 @@ static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
*fmt = format->format;
- fmt = v4l2_subdev_state_get_format(state, TI_CSI2RX_PAD_FIRST_SOURCE,
- format->stream);
+ fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+ format->stream);
+ if (!fmt)
+ return -EINVAL;
+
*fmt = format->format;
return 0;
}
+static int _ti_csi2rx_sd_set_routing(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_krouting *routing)
+{
+ int ret;
+
+ const struct v4l2_mbus_framefmt format = {
+ .width = 640,
+ .height = 480,
+ .code = MEDIA_BUS_FMT_UYVY8_1X16,
+ .field = V4L2_FIELD_NONE,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .ycbcr_enc = V4L2_YCBCR_ENC_601,
+ .quantization = V4L2_QUANTIZATION_LIM_RANGE,
+ .xfer_func = V4L2_XFER_FUNC_SRGB,
+ };
+
+ ret = v4l2_subdev_routing_validate(sd, routing,
+ V4L2_SUBDEV_ROUTING_ONLY_1_TO_1 |
+ V4L2_SUBDEV_ROUTING_NO_SOURCE_MULTIPLEXING);
+
+ if (ret)
+ return ret;
+
+ /* Only stream ID 0 allowed on source pads */
+ for (unsigned int i = 0; i < routing->num_routes; ++i) {
+ const struct v4l2_subdev_route *route = &routing->routes[i];
+
+ if (route->source_stream != 0)
+ return -EINVAL;
+ }
+
+ ret = v4l2_subdev_set_routing_with_fmt(sd, state, routing, &format);
+
+ return ret;
+}
+
+static int ti_csi2rx_sd_set_routing(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ enum v4l2_subdev_format_whence which,
+ struct v4l2_subdev_krouting *routing)
+{
+ struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+
+ if (csi->enable_count > 0)
+ return -EBUSY;
+
+ return _ti_csi2rx_sd_set_routing(sd, state, routing);
+}
+
static int ti_csi2rx_sd_init_state(struct v4l2_subdev *sd,
struct v4l2_subdev_state *state)
{
- struct v4l2_subdev_format format = {
- .pad = TI_CSI2RX_PAD_SINK,
- .format = {
- .width = 640,
- .height = 480,
- .code = MEDIA_BUS_FMT_UYVY8_1X16,
- .field = V4L2_FIELD_NONE,
- .colorspace = V4L2_COLORSPACE_SRGB,
- .ycbcr_enc = V4L2_YCBCR_ENC_601,
- .quantization = V4L2_QUANTIZATION_LIM_RANGE,
- .xfer_func = V4L2_XFER_FUNC_SRGB,
- },
+ struct v4l2_subdev_route routes[] = { {
+ .sink_pad = 0,
+ .sink_stream = 0,
+ .source_pad = TI_CSI2RX_PAD_FIRST_SOURCE,
+ .source_stream = 0,
+ .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+ } };
+
+ struct v4l2_subdev_krouting routing = {
+ .num_routes = 1,
+ .routes = routes,
};
- return ti_csi2rx_sd_set_fmt(sd, state, &format);
+ /* Initialize routing to single route to the fist source pad */
+ return _ti_csi2rx_sd_set_routing(sd, state, &routing);
}
-static int ti_csi2rx_sd_s_stream(struct v4l2_subdev *sd, int enable)
+static int ti_csi2rx_sd_enable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ u32 pad, u64 streams_mask)
{
struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+ struct media_pad *remote_pad;
+ u64 sink_streams;
int ret = 0;
- mutex_lock(&csi->mutex);
+ remote_pad = media_entity_remote_source_pad_unique(&csi->subdev.entity);
+ if (!remote_pad)
+ return -ENODEV;
+ sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+ TI_CSI2RX_PAD_SINK,
+ &streams_mask);
- if (enable) {
- if (csi->enable_count > 0) {
- csi->enable_count++;
- goto out;
- }
+ ret = v4l2_subdev_enable_streams(csi->source, remote_pad->index,
+ sink_streams);
+ if (ret)
+ return ret;
- ret = v4l2_subdev_call(csi->source, video, s_stream, 1);
- if (ret)
- goto out;
+ mutex_lock(&csi->mutex);
+ csi->enable_count++;
+ mutex_unlock(&csi->mutex);
- csi->enable_count++;
- } else {
- if (csi->enable_count == 0) {
- ret = -EINVAL;
- goto out;
- }
+ return 0;
+}
- if (--csi->enable_count > 0)
- goto out;
+static int ti_csi2rx_sd_disable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ u32 pad, u64 streams_mask)
+{
+ struct ti_csi2rx_dev *csi = to_csi2rx_dev(sd);
+ struct media_pad *remote_pad;
+ u64 sink_streams;
+ int ret = 0;
+
+ remote_pad = media_entity_remote_source_pad_unique(&csi->subdev.entity);
+ if (!remote_pad)
+ return -ENODEV;
+ sink_streams = v4l2_subdev_state_xlate_streams(state, pad,
+ TI_CSI2RX_PAD_SINK,
+ &streams_mask);
- ret = v4l2_subdev_call(csi->source, video, s_stream, 0);
+ mutex_lock(&csi->mutex);
+ if (csi->enable_count == 0) {
+ ret = -EINVAL;
+ goto out;
}
+ ret = v4l2_subdev_disable_streams(csi->source, remote_pad->index,
+ sink_streams);
+ if (!ret)
+ --csi->enable_count;
out:
mutex_unlock(&csi->mutex);
return ret;
}
static const struct v4l2_subdev_pad_ops ti_csi2rx_subdev_pad_ops = {
+ .set_routing = ti_csi2rx_sd_set_routing,
.get_fmt = v4l2_subdev_get_fmt,
.set_fmt = ti_csi2rx_sd_set_fmt,
-};
-
-static const struct v4l2_subdev_video_ops ti_csi2rx_subdev_video_ops = {
- .s_stream = ti_csi2rx_sd_s_stream,
+ .enable_streams = ti_csi2rx_sd_enable_streams,
+ .disable_streams = ti_csi2rx_sd_disable_streams,
};
static const struct v4l2_subdev_ops ti_csi2rx_subdev_ops = {
- .video = &ti_csi2rx_subdev_video_ops,
.pad = &ti_csi2rx_subdev_pad_ops,
};
@@ -1244,7 +1364,7 @@ static int ti_csi2rx_v4l2_init(struct ti_csi2rx_dev *csi)
v4l2_subdev_init(sd, &ti_csi2rx_subdev_ops);
sd->internal_ops = &ti_csi2rx_internal_ops;
sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
- sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
strscpy(sd->name, dev_name(csi->dev), sizeof(sd->name));
sd->dev = csi->dev;
sd->entity.ops = &ti_csi2rx_subdev_entity_ops;
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* [PATCH v3 12/13] media: ti: j721e-csi2rx: Submit all available buffers
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (10 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 11/13] media: ti: j721e-csi2rx: add multistream support Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-17 6:55 ` [PATCH v3 13/13] media: ti: j721e-csi2rx: Change the drain architecture for multistream Rishikesh Donadkar
12 siblings, 0 replies; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
From: Jai Luthra <j-luthra@ti.com>
We already make sure to submit all available buffers to DMA in each DMA
completion callback.
Move that logic in a separate function, and use it during stream start
as well, as most application queue all their buffers before stream on.
Signed-off-by: Jai Luthra <j-luthra@ti.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 43 +++++++++++--------
1 file changed, 24 insertions(+), 19 deletions(-)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index d00b5a1848043..23d63d8bcd36a 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -644,6 +644,27 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
return ret;
}
+static int ti_csi2rx_dma_submit_pending(struct ti_csi2rx_ctx *ctx)
+{
+ struct ti_csi2rx_dma *dma = &ctx->dma;
+ struct ti_csi2rx_buffer *buf;
+ int ret = 0;
+
+ /* If there are more buffers to process then start their transfer. */
+ while (!list_empty(&dma->queue)) {
+ buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
+ ret = ti_csi2rx_start_dma(ctx, buf);
+ if (ret) {
+ dev_err(ctx->csi->dev,
+ "Failed to queue the next buffer for DMA\n");
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ break;
+ }
+ list_move_tail(&buf->list, &dma->submitted);
+ }
+ return ret;
+}
+
static void ti_csi2rx_dma_callback(void *param)
{
struct ti_csi2rx_buffer *buf = param;
@@ -664,18 +685,7 @@ static void ti_csi2rx_dma_callback(void *param)
vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
list_del(&buf->list);
- /* If there are more buffers to process then start their transfer. */
- while (!list_empty(&dma->queue)) {
- buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
-
- if (ti_csi2rx_start_dma(ctx, buf)) {
- dev_err(ctx->csi->dev,
- "Failed to queue the next buffer for DMA\n");
- vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
- } else {
- list_move_tail(&buf->list, &dma->submitted);
- }
- }
+ ti_csi2rx_dma_submit_pending(ctx);
if (list_empty(&dma->submitted))
dma->state = TI_CSI2RX_DMA_IDLE;
@@ -884,7 +894,6 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vq);
struct ti_csi2rx_dev *csi = ctx->csi;
struct ti_csi2rx_dma *dma = &ctx->dma;
- struct ti_csi2rx_buffer *buf;
struct v4l2_subdev_krouting *routing;
struct v4l2_subdev_route *route = NULL, *r;
struct media_pad *remote_pad;
@@ -948,16 +957,13 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
ctx->sequence = 0;
spin_lock_irqsave(&dma->lock, flags);
- buf = list_entry(dma->queue.next, struct ti_csi2rx_buffer, list);
- ret = ti_csi2rx_start_dma(ctx, buf);
+ ret = ti_csi2rx_dma_submit_pending(ctx);
if (ret) {
- dev_err(csi->dev, "Failed to start DMA: %d\n", ret);
spin_unlock_irqrestore(&dma->lock, flags);
- goto err_pipeline;
+ goto err_dma;
}
- list_move_tail(&buf->list, &dma->submitted);
dma->state = TI_CSI2RX_DMA_ACTIVE;
spin_unlock_irqrestore(&dma->lock, flags);
@@ -972,7 +978,6 @@ static int ti_csi2rx_start_streaming(struct vb2_queue *vq, unsigned int count)
err_dma:
ti_csi2rx_stop_dma(ctx);
-err_pipeline:
video_device_pipeline_stop(&ctx->vdev);
writel(0, csi->shim + SHIM_CNTL);
writel(0, csi->shim + SHIM_DMACNTX(ctx->idx));
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* [PATCH v3 13/13] media: ti: j721e-csi2rx: Change the drain architecture for multistream
2025-04-17 6:55 [PATCH v3 00/13] media: cadence,ti: CSI2RX Multistream Support Rishikesh Donadkar
` (11 preceding siblings ...)
2025-04-17 6:55 ` [PATCH v3 12/13] media: ti: j721e-csi2rx: Submit all available buffers Rishikesh Donadkar
@ 2025-04-17 6:55 ` Rishikesh Donadkar
2025-04-25 11:35 ` Jai Luthra
12 siblings, 1 reply; 36+ messages in thread
From: Rishikesh Donadkar @ 2025-04-17 6:55 UTC (permalink / raw)
To: jai.luthra, mripard
Cc: linux-kernel, linux-media, devicetree, r-donadkar, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
In Multistream use cases, we may face buffer starvation due to various
reasons like slow downstream element in gstreamer pipeline. In these
scenarios we need to make sure that the data corresponding to the slow
pipeline is pulled out of the shared HW FIFO in CSI2RX IP to prevent
other streams to get stalled due to FIFO overflow.
Previously, in case of buffer starvation, dma was marked IDLE and the
next buffer_queue() would drain the data before marking new buffer ready
for DMA transaction. Here the driver waits for the next VIDIOC_QBUF
ioctl callback to drain the stale data from the HW FIFO, if there is a
delay in this callback being called, HW FIFO will overflow leading all
the other camera pipelines in the media graph to hang.
Introduce a new architecture where, CSI data is always pulled out of
the shared HW FIFO irrespective of the availability of buffers from
userspace.
Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
---
.../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 96 +++++++------------
1 file changed, 33 insertions(+), 63 deletions(-)
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index 23d63d8bcd36a..7f476c78c4a92 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -57,7 +57,6 @@
#define TI_CSI2RX_MAX_SOURCE_PADS TI_CSI2RX_MAX_CTX
#define TI_CSI2RX_MAX_PADS (1 + TI_CSI2RX_MAX_SOURCE_PADS)
-#define DRAIN_TIMEOUT_MS 50
#define DRAIN_BUFFER_SIZE SZ_32K
struct ti_csi2rx_fmt {
@@ -77,7 +76,6 @@ struct ti_csi2rx_buffer {
enum ti_csi2rx_dma_state {
TI_CSI2RX_DMA_STOPPED, /* Streaming not started yet. */
- TI_CSI2RX_DMA_IDLE, /* Streaming but no pending DMA operation. */
TI_CSI2RX_DMA_ACTIVE, /* Streaming and pending DMA operation. */
};
@@ -238,6 +236,10 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
struct ti_csi2rx_buffer *buf);
+/* Forward declarations needed by ti_csi2rx_drain_callback. */
+static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx);
+static int ti_csi2rx_dma_submit_pending(struct ti_csi2rx_ctx *ctx);
+
static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
{
unsigned int i;
@@ -589,9 +591,28 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
static void ti_csi2rx_drain_callback(void *param)
{
- struct completion *drain_complete = param;
+ struct ti_csi2rx_ctx *ctx = param;
+ struct ti_csi2rx_dma *dma = &ctx->dma;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dma->lock, flags);
- complete(drain_complete);
+ if (dma->state == TI_CSI2RX_DMA_STOPPED) {
+ spin_unlock_irqrestore(&dma->lock, flags);
+ return;
+ }
+
+ /*
+ * If dma->queue is empty, it signals no buffer has arrived from
+ * user space, so, queue more transaction to drain dma
+ */
+ if (list_empty(&dma->queue)) {
+ if (ti_csi2rx_drain_dma(ctx))
+ dev_warn(ctx->csi->dev, "DMA drain failed\n");
+ } else {
+ ti_csi2rx_dma_submit_pending(ctx);
+ }
+ spin_unlock_irqrestore(&dma->lock, flags);
}
/*
@@ -609,12 +630,9 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
{
struct ti_csi2rx_dev *csi = ctx->csi;
struct dma_async_tx_descriptor *desc;
- struct completion drain_complete;
dma_cookie_t cookie;
int ret;
- init_completion(&drain_complete);
-
desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
csi->drain.len, DMA_DEV_TO_MEM,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
@@ -624,7 +642,7 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
}
desc->callback = ti_csi2rx_drain_callback;
- desc->callback_param = &drain_complete;
+ desc->callback_param = ctx;
cookie = dmaengine_submit(desc);
ret = dma_submit_error(cookie);
@@ -633,13 +651,6 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
dma_async_issue_pending(ctx->dma.chan);
- if (!wait_for_completion_timeout(&drain_complete,
- msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
- dmaengine_terminate_sync(ctx->dma.chan);
- dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
- ret = -ETIMEDOUT;
- goto out;
- }
out:
return ret;
}
@@ -687,9 +698,11 @@ static void ti_csi2rx_dma_callback(void *param)
ti_csi2rx_dma_submit_pending(ctx);
- if (list_empty(&dma->submitted))
- dma->state = TI_CSI2RX_DMA_IDLE;
-
+ if (list_empty(&dma->submitted)) {
+ if (ti_csi2rx_drain_dma(ctx))
+ dev_warn(ctx->csi->dev,
+ "DMA drain failed on one of the transactions\n");
+ }
spin_unlock_irqrestore(&dma->lock, flags);
}
@@ -742,7 +755,7 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
* enforced before terminating DMA.
*/
ret = ti_csi2rx_drain_dma(ctx);
- if (ret && ret != -ETIMEDOUT)
+ if (ret)
dev_warn(ctx->csi->dev,
"Failed to drain DMA. Next frame might be bogus\n");
}
@@ -809,57 +822,14 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
struct ti_csi2rx_buffer *buf;
struct ti_csi2rx_dma *dma = &ctx->dma;
- bool restart_dma = false;
unsigned long flags = 0;
- int ret;
buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
buf->ctx = ctx;
spin_lock_irqsave(&dma->lock, flags);
- /*
- * Usually the DMA callback takes care of queueing the pending buffers.
- * But if DMA has stalled due to lack of buffers, restart it now.
- */
- if (dma->state == TI_CSI2RX_DMA_IDLE) {
- /*
- * Do not restart DMA with the lock held because
- * ti_csi2rx_drain_dma() might block for completion.
- * There won't be a race on queueing DMA anyway since the
- * callback is not being fired.
- */
- restart_dma = true;
- dma->state = TI_CSI2RX_DMA_ACTIVE;
- } else {
- list_add_tail(&buf->list, &dma->queue);
- }
+ list_add_tail(&buf->list, &dma->queue);
spin_unlock_irqrestore(&dma->lock, flags);
-
- if (restart_dma) {
- /*
- * Once frames start dropping, some data gets stuck in the DMA
- * pipeline somewhere. So the first DMA transfer after frame
- * drops gives a partial frame. This is obviously not useful to
- * the application and will only confuse it. Issue a DMA
- * transaction to drain that up.
- */
- ret = ti_csi2rx_drain_dma(ctx);
- if (ret && ret != -ETIMEDOUT)
- dev_warn(ctx->csi->dev,
- "Failed to drain DMA. Next frame might be bogus\n");
-
- spin_lock_irqsave(&dma->lock, flags);
- ret = ti_csi2rx_start_dma(ctx, buf);
- if (ret) {
- vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
- dma->state = TI_CSI2RX_DMA_IDLE;
- spin_unlock_irqrestore(&dma->lock, flags);
- dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
- } else {
- list_add_tail(&buf->list, &dma->submitted);
- spin_unlock_irqrestore(&dma->lock, flags);
- }
- }
}
static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
--
2.34.1
^ permalink raw reply [flat|nested] 36+ messages in thread* Re: [PATCH v3 13/13] media: ti: j721e-csi2rx: Change the drain architecture for multistream
2025-04-17 6:55 ` [PATCH v3 13/13] media: ti: j721e-csi2rx: Change the drain architecture for multistream Rishikesh Donadkar
@ 2025-04-25 11:35 ` Jai Luthra
0 siblings, 0 replies; 36+ messages in thread
From: Jai Luthra @ 2025-04-25 11:35 UTC (permalink / raw)
To: Rishikesh Donadkar
Cc: mripard, linux-kernel, linux-media, devicetree, devarsht,
y-abhilashchandra, mchehab, robh, krzk+dt, conor+dt, vaishnav.a,
s-jain1, vigneshr, sakari.ailus, hverkuil-cisco, tomi.valkeinen,
jai.luthra, changhuang.liang, jack.zhu, laurent.pinchart
[-- Attachment #1: Type: text/plain, Size: 9825 bytes --]
Hi Rishikesh,
Thanks for the patch.
On Thu, Apr 17, 2025 at 12:25:54PM +0530, Rishikesh Donadkar wrote:
> In Multistream use cases, we may face buffer starvation due to various
> reasons like slow downstream element in gstreamer pipeline. In these
> scenarios we need to make sure that the data corresponding to the slow
> pipeline is pulled out of the shared HW FIFO in CSI2RX IP to prevent
> other streams to get stalled due to FIFO overflow.
>
> Previously, in case of buffer starvation, dma was marked IDLE and the
> next buffer_queue() would drain the data before marking new buffer ready
> for DMA transaction. Here the driver waits for the next VIDIOC_QBUF
> ioctl callback to drain the stale data from the HW FIFO, if there is a
> delay in this callback being called, HW FIFO will overflow leading all
> the other camera pipelines in the media graph to hang.
The above description is a little hard to follow, and not correct. A "late"
QBUF callback is what buffer starvation *is*. With single-stream scenarios the
older drain architecture worked fine, as the goal there was to drain out stale
data in the FIFOs when buffers are available again.
>
> Introduce a new architecture where, CSI data is always pulled out of
> the shared HW FIFO irrespective of the availability of buffers from
> userspace.
I think this description does not make it very clear why multiple streams
(VCs) cause the older drain architecture to not work, and what exactly is the
limitation in the hardware FIFOs.
How about the following instead:
On buffer starvation the DMA is marked IDLE, and the stale data in the
internal FIFOs gets drained only on the next VIDIOC_QBUF call from the
userspace. This approach works fine for a single stream case.
But in multistream scenarios, buffer starvation for one stream i.e. one
virtual channel, can block the shared HW FIFO of the CSI2RX IP. This can
stall the pipeline for all other virtual channels, even if buffers are
available for them.
This patch introduces a new architecture, that continuously drains data
from the shared HW FIFO into a small (32KiB) buffer if no buffers are made
available to the driver from the userspace. This ensures independence
between different streams, where a slower downstream element for one
camera does not block streaming for other cameras.
>
> Signed-off-by: Rishikesh Donadkar <r-donadkar@ti.com>
> ---
> .../platform/ti/j721e-csi2rx/j721e-csi2rx.c | 96 +++++++------------
> 1 file changed, 33 insertions(+), 63 deletions(-)
>
> diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> index 23d63d8bcd36a..7f476c78c4a92 100644
> --- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> +++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
> @@ -57,7 +57,6 @@
> #define TI_CSI2RX_MAX_SOURCE_PADS TI_CSI2RX_MAX_CTX
> #define TI_CSI2RX_MAX_PADS (1 + TI_CSI2RX_MAX_SOURCE_PADS)
>
> -#define DRAIN_TIMEOUT_MS 50
> #define DRAIN_BUFFER_SIZE SZ_32K
As you are draining 32KiB chunks, it is likely that the hardware is in the
middle of a frame when the userspace queues a new buffer and you stop
draining, and change the target address to a user-facing buffer. How does the
PSIL/DMA engine handle the SoF/EoF boundaries in such a case?
Would it be possible for you to verify if the first user-facing buffer after
draining has valid data? And if not, please highlight it so it may get fixed
in the future.
>
> struct ti_csi2rx_fmt {
> @@ -77,7 +76,6 @@ struct ti_csi2rx_buffer {
>
> enum ti_csi2rx_dma_state {
> TI_CSI2RX_DMA_STOPPED, /* Streaming not started yet. */
> - TI_CSI2RX_DMA_IDLE, /* Streaming but no pending DMA operation. */
> TI_CSI2RX_DMA_ACTIVE, /* Streaming and pending DMA operation. */
> };
>
> @@ -238,6 +236,10 @@ static const struct ti_csi2rx_fmt ti_csi2rx_formats[] = {
> static int ti_csi2rx_start_dma(struct ti_csi2rx_ctx *ctx,
> struct ti_csi2rx_buffer *buf);
>
> +/* Forward declarations needed by ti_csi2rx_drain_callback. */
> +static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx);
> +static int ti_csi2rx_dma_submit_pending(struct ti_csi2rx_ctx *ctx);
> +
> static const struct ti_csi2rx_fmt *find_format_by_fourcc(u32 pixelformat)
> {
> unsigned int i;
> @@ -589,9 +591,28 @@ static void ti_csi2rx_setup_shim(struct ti_csi2rx_ctx *ctx)
>
> static void ti_csi2rx_drain_callback(void *param)
> {
> - struct completion *drain_complete = param;
> + struct ti_csi2rx_ctx *ctx = param;
> + struct ti_csi2rx_dma *dma = &ctx->dma;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&dma->lock, flags);
>
> - complete(drain_complete);
> + if (dma->state == TI_CSI2RX_DMA_STOPPED) {
> + spin_unlock_irqrestore(&dma->lock, flags);
> + return;
> + }
> +
> + /*
> + * If dma->queue is empty, it signals no buffer has arrived from
> + * user space, so, queue more transaction to drain dma
> + */
> + if (list_empty(&dma->queue)) {
> + if (ti_csi2rx_drain_dma(ctx))
> + dev_warn(ctx->csi->dev, "DMA drain failed\n");
> + } else {
> + ti_csi2rx_dma_submit_pending(ctx);
> + }
> + spin_unlock_irqrestore(&dma->lock, flags);
> }
>
> /*
> @@ -609,12 +630,9 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
> {
> struct ti_csi2rx_dev *csi = ctx->csi;
> struct dma_async_tx_descriptor *desc;
> - struct completion drain_complete;
> dma_cookie_t cookie;
> int ret;
>
> - init_completion(&drain_complete);
> -
> desc = dmaengine_prep_slave_single(ctx->dma.chan, csi->drain.paddr,
> csi->drain.len, DMA_DEV_TO_MEM,
> DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
> @@ -624,7 +642,7 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
> }
>
> desc->callback = ti_csi2rx_drain_callback;
> - desc->callback_param = &drain_complete;
> + desc->callback_param = ctx;
>
> cookie = dmaengine_submit(desc);
> ret = dma_submit_error(cookie);
> @@ -633,13 +651,6 @@ static int ti_csi2rx_drain_dma(struct ti_csi2rx_ctx *ctx)
>
> dma_async_issue_pending(ctx->dma.chan);
>
> - if (!wait_for_completion_timeout(&drain_complete,
> - msecs_to_jiffies(DRAIN_TIMEOUT_MS))) {
> - dmaengine_terminate_sync(ctx->dma.chan);
> - dev_dbg(csi->dev, "DMA transfer timed out for drain buffer\n");
> - ret = -ETIMEDOUT;
> - goto out;
> - }
> out:
> return ret;
> }
> @@ -687,9 +698,11 @@ static void ti_csi2rx_dma_callback(void *param)
>
> ti_csi2rx_dma_submit_pending(ctx);
>
> - if (list_empty(&dma->submitted))
> - dma->state = TI_CSI2RX_DMA_IDLE;
> -
> + if (list_empty(&dma->submitted)) {
> + if (ti_csi2rx_drain_dma(ctx))
> + dev_warn(ctx->csi->dev,
> + "DMA drain failed on one of the transactions\n");
Checkpatch warning here:
CHECK: Alignment should match open parenthesis
#138: FILE: drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c:704:
+ dev_warn(ctx->csi->dev,
+ "DMA drain failed on one of the transactions\n");
total: 0 errors, 0 warnings, 1 checks, 167 lines checked
> + }
> spin_unlock_irqrestore(&dma->lock, flags);
> }
>
> @@ -742,7 +755,7 @@ static void ti_csi2rx_stop_dma(struct ti_csi2rx_ctx *ctx)
> * enforced before terminating DMA.
> */
> ret = ti_csi2rx_drain_dma(ctx);
> - if (ret && ret != -ETIMEDOUT)
> + if (ret)
> dev_warn(ctx->csi->dev,
> "Failed to drain DMA. Next frame might be bogus\n");
> }
> @@ -809,57 +822,14 @@ static void ti_csi2rx_buffer_queue(struct vb2_buffer *vb)
> struct ti_csi2rx_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> struct ti_csi2rx_buffer *buf;
> struct ti_csi2rx_dma *dma = &ctx->dma;
> - bool restart_dma = false;
> unsigned long flags = 0;
> - int ret;
>
> buf = container_of(vb, struct ti_csi2rx_buffer, vb.vb2_buf);
> buf->ctx = ctx;
>
> spin_lock_irqsave(&dma->lock, flags);
> - /*
> - * Usually the DMA callback takes care of queueing the pending buffers.
> - * But if DMA has stalled due to lack of buffers, restart it now.
> - */
> - if (dma->state == TI_CSI2RX_DMA_IDLE) {
> - /*
> - * Do not restart DMA with the lock held because
> - * ti_csi2rx_drain_dma() might block for completion.
> - * There won't be a race on queueing DMA anyway since the
> - * callback is not being fired.
> - */
> - restart_dma = true;
> - dma->state = TI_CSI2RX_DMA_ACTIVE;
> - } else {
> - list_add_tail(&buf->list, &dma->queue);
> - }
> + list_add_tail(&buf->list, &dma->queue);
> spin_unlock_irqrestore(&dma->lock, flags);
> -
> - if (restart_dma) {
> - /*
> - * Once frames start dropping, some data gets stuck in the DMA
> - * pipeline somewhere. So the first DMA transfer after frame
> - * drops gives a partial frame. This is obviously not useful to
> - * the application and will only confuse it. Issue a DMA
> - * transaction to drain that up.
> - */
> - ret = ti_csi2rx_drain_dma(ctx);
> - if (ret && ret != -ETIMEDOUT)
> - dev_warn(ctx->csi->dev,
> - "Failed to drain DMA. Next frame might be bogus\n");
> -
> - spin_lock_irqsave(&dma->lock, flags);
> - ret = ti_csi2rx_start_dma(ctx, buf);
> - if (ret) {
> - vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> - dma->state = TI_CSI2RX_DMA_IDLE;
> - spin_unlock_irqrestore(&dma->lock, flags);
> - dev_err(ctx->csi->dev, "Failed to start DMA: %d\n", ret);
> - } else {
> - list_add_tail(&buf->list, &dma->submitted);
> - spin_unlock_irqrestore(&dma->lock, flags);
> - }
> - }
> }
>
> static int ti_csi2rx_get_vc(struct ti_csi2rx_ctx *ctx)
> --
> 2.34.1
>
Thanks,
Jai
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 36+ messages in thread