[PATCH 0/4] media: rcar-isp: Add NV12M support and fix stride

[PATCH 0/4] media: rcar-isp: Add NV12M support and fix stride

[Niklas Söderlund]

Hello,

The main part of this series aims to add NV12M output format support and 
there is a small drive-by fix to honor the image stride from input 
images in patch 1/4.

Patch 2/4 address a confusion around how much data was needed to flush 
out all pixels to the output buffer. Later versions of the datasheet 
makes it use clear and how it relates to image strides. This is needed 
for NV12M support as we need to tightly control it to generate valid 
NV12M frames.

Patch 3/4 is a quality of life fix where the current driver looks at the 
pixelformat to figure out how many planes are used. As it only supports 
RGB (1 plane) and NV16M (2 plane) this worked, but before adding NV12M 
(2 plane) look at the correct field so we don't have to maintain a list 
of pixelformats that have two planes ourself.

Finally patch 4/4 adds NV12M support. Enabling support for it is 
straightforward but one snag around how the DMA-engine works.

When the DMA-engine is configured to only output CbCr data from odd or
even lines it still outputs blanks pixels to the buffer for the line
where no data is taken.

To work around this and still produce valid NV12 frames the device needs
to be configured so all blank pixels written to the buffer are part of
the line padding. In addition to also keeping the same number of padding
bytes as the Y plane.

The DMA-engine is able to do this as we can set different strides for
the two planes and haw two DMA-engine lines (one with data + some
padding and one with only padding) make up a single correctly formatted
NV12 CbCr line.

For readers comparing the DMA configuration with the datasheet this
driver takes the CbCr data from even lines, compared to odd lines in the
datasheet example. The reason for this is that if odd lines are used the
first even line adds half a NV12 line of padding at the start of the
CbCr plane. To counter this the CbCr plane would need a padding line at
the start of the plane which is not supported, or we would need to set
the plane start address half a line before the buffer actually starts
and potentially corrupt memory as we write blank bytes outside our
buffer.

Tested on V4H SparrowHawk without any regressions and the output frames 
looks good.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
Niklas Söderlund (3):
      media: rcar-isp: ispcore: Update image data flush settings
      media: rcar-isp: ispcore: Look at number of planes when programming
      media: rcar-isp: ispcore: Add NV12M support

Yuya Hamamachi (1):
      media: rcar-isp: ispcore: Honour requested bytesperline on RAW input

 drivers/media/platform/renesas/rcar-isp/core-io.c | 78 +++++++++++++++++++++--
 drivers/media/platform/renesas/rcar-isp/core.c    | 45 ++++++++++++-
 2 files changed, 115 insertions(+), 8 deletions(-)
---
base-commit: d7e7bca7169873b46e857c025c10f3336c1258d8
change-id: 20261015-v4l2-add-nv12-9358df2bf649

Best regards,
--  
Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>

[PATCH 1/4] media: rcar-isp: ispcore: Honour requested bytesperline on RAW input

[Niklas Söderlund]

From: Yuya Hamamachi <yuya.hamamachi.sx@renesas.com>

The RAW input video device always overwrote the bytesperline requested
by userspace with width * bpp. The input buffers are however shared with
the VIN, which may use a larger line stride (for example 3296 instead of
3280 bytes for a 1640 pixel wide 10 bit image). The dmabuf exported by
the ISP was then smaller than the VIN buffer size, and queueing it to
the VIN failed with -EINVAL:

  __prepare_dmabuf: invalid dmabuf length 4042752 for plane 0,
  minimum length 4060672

Use the requested bytesperline if it is larger than the minimum.

Signed-off-by: Yuya Hamamachi <yuya.hamamachi.sx@renesas.com>
[Niklas: Use max(min(..)) instead of if statement]
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/renesas/rcar-isp/core-io.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/core-io.c b/drivers/media/platform/renesas/rcar-isp/core-io.c
index b60b91f43d42..badde72352c1 100644
--- a/drivers/media/platform/renesas/rcar-isp/core-io.c
+++ b/drivers/media/platform/renesas/rcar-isp/core-io.c
@@ -5,6 +5,8 @@
  * Copyright (C) 2026 Ragnatech AB
  */
 
+#include <linux/sizes.h>
+
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-event.h>
 #include <media/v4l2-ioctl.h>
@@ -335,6 +337,7 @@ static void risp_io_input_try_format(struct rcar_isp_core_io *io,
 				     struct v4l2_pix_format_mplane *pix)
 {
 	unsigned int bpp = 0;
+	unsigned int bpl;
 
 	v4l_bound_align_image(&pix->width, RISP_MIN_WIDTH, RISP_MAX_WIDTH, RISP_SIZE_ALIGNMENT,
 			      &pix->height, RISP_MIN_HEIGHT, RISP_MAX_HEIGHT, RISP_SIZE_ALIGNMENT,
@@ -352,11 +355,17 @@ static void risp_io_input_try_format(struct rcar_isp_core_io *io,
 		bpp = risp_io_input_formats[0].bpp;
 	}
 
+	/*
+	 * Honour the line stride requested by userspace, the buffers are
+	 * shared with the VIN which may use a larger stride than the minimum.
+	 */
+	bpl = max(pix->width * bpp, min(pix->plane_fmt[0].bytesperline, SZ_64K));
+
 	pix->field = V4L2_FIELD_NONE;
 	pix->colorspace = V4L2_COLORSPACE_RAW;
 
 	pix->num_planes = 1;
-	pix->plane_fmt[0].bytesperline = pix->width * bpp;
+	pix->plane_fmt[0].bytesperline = bpl;
 	pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
 }
 

-- 
2.55.0

[PATCH 2/4] media: rcar-isp: ispcore: Update image data flush settings

[Niklas Söderlund]

Later versions of the datasheet have an updated chapter on how the image
out flush settings depends on proper stride settings. Update the bytes
per line calculation and flush out settings to match the datasheet
clarification.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/renesas/rcar-isp/core-io.c | 28 ++++++++++++++++++++---
 drivers/media/platform/renesas/rcar-isp/core.c    |  2 +-
 2 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/core-io.c b/drivers/media/platform/renesas/rcar-isp/core-io.c
index badde72352c1..48db37bad69c 100644
--- a/drivers/media/platform/renesas/rcar-isp/core-io.c
+++ b/drivers/media/platform/renesas/rcar-isp/core-io.c
@@ -733,6 +733,28 @@ static const struct v4l2_pix_format_mplane risp_io_capture_default_format = {
 	},
 };
 
+static unsigned int risp_io_calture_bytesperline(unsigned int width, unsigned int bpp)
+{
+	/*
+	 * The datasheet specifies strict alignment and padding rules to be able
+	 * to flush out all the pixels. Each line must be 256 byte aligned and
+	 * at the end of the frame at least one extra line of space must be
+	 * available.
+	 *
+	 * As we can't extend the buffer with one extra line we need to work
+	 * with stride setting to satisfy both constraints. Fortunately the
+	 * datasheet gives us a calculation for this.
+	 *
+	 *   FLUSH[pixels] = ALIGN(WIDTH[pixels], 256)
+	 *
+	 *   PADDING[bytes] = ALIGN(BPP[bytes] * WIDTH[pixels], 256) - BPP[bytes] * WIDTH[pixels] + BPP[bytes] * FLUSH[pixels]
+	 */
+	unsigned int data = bpp * width;
+	unsigned int padding = ALIGN(data, 256) - data + bpp * ALIGN(width, 256);
+
+	return data + padding;
+}
+
 static void risp_io_capture_try_format(struct rcar_isp_core_io *io,
 				       struct v4l2_pix_format_mplane *pix)
 {
@@ -749,16 +771,16 @@ static void risp_io_capture_try_format(struct rcar_isp_core_io *io,
 	case V4L2_PIX_FMT_NV16M:
 		pix->quantization = V4L2_QUANTIZATION_LIM_RANGE;
 		pix->num_planes = 2;
-		pix->plane_fmt[0].bytesperline = ALIGN(pix->width, 256);
+		pix->plane_fmt[0].bytesperline = risp_io_calture_bytesperline(pix->width, 1);
 		pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
-		pix->plane_fmt[1].bytesperline = ALIGN(pix->width, 256);
+		pix->plane_fmt[1].bytesperline = pix->plane_fmt[0].bytesperline;
 		pix->plane_fmt[1].sizeimage = pix->plane_fmt[1].bytesperline * pix->height;
 		break;
 	default:
 		pix->pixelformat = V4L2_PIX_FMT_XBGR32;
 		pix->quantization = V4L2_QUANTIZATION_FULL_RANGE;
 		pix->num_planes = 1;
-		pix->plane_fmt[0].bytesperline = ALIGN(pix->width * 4, 256);
+		pix->plane_fmt[0].bytesperline = risp_io_calture_bytesperline(pix->width, 4);
 		pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
 		break;
 	}
diff --git a/drivers/media/platform/renesas/rcar-isp/core.c b/drivers/media/platform/renesas/rcar-isp/core.c
index b5861d0cd0e8..3e135e84b75b 100644
--- a/drivers/media/platform/renesas/rcar-isp/core.c
+++ b/drivers/media/platform/renesas/rcar-isp/core.c
@@ -398,7 +398,7 @@ static int risp_core_config_output(struct rcar_isp_core *core,
 
 	/* Set image out flush EOF. */
 	risp_core_write(core, ISPCORE_DMA_IMAGE_FLUSH_OUT_REG(index),
-			pix->plane_fmt[0].bytesperline <<
+			ALIGN(pix->width, 256) <<
 			ISPCORE_DMA_IMAGE_FLUSH_OUT_PADDING_PIXEL_EOF_SHIFT);
 
 	/* Enable DMA and set burst length. */

-- 
2.55.0

[PATCH 3/4] media: rcar-isp: ispcore: Look at number of planes when programming

[Niklas Söderlund]

Instead of expanding a list of formats that uses two planes look at the
format information directly.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/renesas/rcar-isp/core.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/core.c b/drivers/media/platform/renesas/rcar-isp/core.c
index 3e135e84b75b..cb9d9033d0cf 100644
--- a/drivers/media/platform/renesas/rcar-isp/core.c
+++ b/drivers/media/platform/renesas/rcar-isp/core.c
@@ -126,8 +126,7 @@ static void risp_core_job_run_output(struct rcar_isp_core *core,
 		mem = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
 		risp_core_write(core, reg, mem);
 
-		/* Only NV16 uses 2 planes. */
-		if (fmt->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_NV16M)
+		if (fmt->fmt.pix_mp.num_planes != 2)
 			continue;
 
 		reg = ISPCORE_DMA_IMAGE_FRAME_BASE_ADDRESS_COMP1(0, frame);

-- 
2.55.0

[PATCH 4/4] media: rcar-isp: ispcore: Add NV12M support

[Niklas Söderlund]

It is possible to support NV12M output by only outputting the CbCr plane
from even or odd lines. Enabling support for it is straightforward but
one snag around how the DMA-engine works.

When the DMA-engine is configured to only output CbCr data from odd or
even lines it still outputs blanks pixels to the buffer for the line
where no data is taken.

To work around this and still produce valid NV12 frames the device needs
to be configured so all blank pixels written to the buffer are part of
the line padding. In addition to also keeping the same number of padding
bytes as the Y plane.

The DMA-engine is able to do this as we can set different strides for
the two planes and haw two DMA-engine lines (one with data + some
padding and one with only padding) make up a single correctly formatted
NV12 CbCr line.

For readers comparing the DMA configuration with the datasheet this
driver takes the CbCr data from even lines, compared to odd lines in the
datasheet example. The reason for this is that if odd lines are used the
first even line adds half a NV12 line of padding at the start of the
CbCr plane. To counter this the CbCr plane would need a padding line at
the start of the plane which is not supported, or we would need to set
the plane start address half a line before the buffer actually starts
and potentially corrupt memory as we write blank bytes outside our
buffer.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/renesas/rcar-isp/core-io.c | 39 +++++++++++++++++++++-
 drivers/media/platform/renesas/rcar-isp/core.c    | 40 +++++++++++++++++++++++
 2 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/core-io.c b/drivers/media/platform/renesas/rcar-isp/core-io.c
index 48db37bad69c..a9c522212e9e 100644
--- a/drivers/media/platform/renesas/rcar-isp/core-io.c
+++ b/drivers/media/platform/renesas/rcar-isp/core-io.c
@@ -768,6 +768,39 @@ static void risp_io_capture_try_format(struct rcar_isp_core_io *io,
 	pix->xfer_func = V4L2_XFER_FUNC_SRGB;
 
 	switch (pix->pixelformat) {
+	case V4L2_PIX_FMT_NV12M:
+		pix->quantization = V4L2_QUANTIZATION_LIM_RANGE;
+		pix->num_planes = 2;
+		/*
+		 * The DMA-engine still outputs pixels for the height lines of
+		 * the CrCb plane. To produce a NV12 frame we must ensure all
+		 * these blank pixels are in the padding area of the frame.
+		 *
+		 * To do this the stride of the CrCb plane need to be half that
+		 * of the Y plane. The first "DMA-engine line" will write the
+		 * CrCb data + some blank padding, while the second "DMA-engine
+		 * line" will only write blank padding bytes. Two "DMA-engine
+		 * lines" will then create one correct NV12 line in the CrCb
+		 * plane.
+		 *
+		 * All frame data also needs to fits in the first half of the
+		 * line. The strict DMA flush settings used already ensures
+		 * this is always the case.
+		 *
+		 * Finally the NV12 stride of the Y and CrCb plane must be the
+		 * same and the R-Car ISP needs the hardware stride to be 256
+		 * byte aligned. By align the Y plane lines to 512 bytes the
+		 * CrCb planes hardware stride is always aligned with 256 when
+		 * cut in half.
+		 */
+		pix->plane_fmt[0].bytesperline =
+			ALIGN(risp_io_calture_bytesperline(pix->width, 1), 512);
+		pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
+		/* Both planes must have the same length. */
+		pix->plane_fmt[1].bytesperline = pix->plane_fmt[0].bytesperline;
+		/* The CrCb plane is half height of the Y plane. */
+		pix->plane_fmt[1].sizeimage = pix->plane_fmt[1].bytesperline * pix->height / 2;
+		break;
 	case V4L2_PIX_FMT_NV16M:
 		pix->quantization = V4L2_QUANTIZATION_LIM_RANGE;
 		pix->num_planes = 2;
@@ -794,9 +827,12 @@ static int risp_io_capture_enum_fmt(struct file *file, void *priv,
 
 	switch (f->index) {
 	case 0:
-		f->pixelformat = V4L2_PIX_FMT_NV16M;
+		f->pixelformat = V4L2_PIX_FMT_NV12M;
 		break;
 	case 1:
+		f->pixelformat = V4L2_PIX_FMT_NV16M;
+		break;
+	case 2:
 		f->pixelformat = V4L2_PIX_FMT_XBGR32;
 		break;
 	default:
@@ -854,6 +890,7 @@ static int risp_io_capture_enum_framesizes(struct file *file, void *fh,
 		return -EINVAL;
 
 	switch (fsize->pixel_format) {
+	case V4L2_PIX_FMT_NV12M:
 	case V4L2_PIX_FMT_NV16M:
 	case V4L2_PIX_FMT_XBGR32:
 		break;
diff --git a/drivers/media/platform/renesas/rcar-isp/core.c b/drivers/media/platform/renesas/rcar-isp/core.c
index cb9d9033d0cf..9be2f73394ee 100644
--- a/drivers/media/platform/renesas/rcar-isp/core.c
+++ b/drivers/media/platform/renesas/rcar-isp/core.c
@@ -341,6 +341,45 @@ static int risp_core_config_output(struct rcar_isp_core *core,
 	/* For all frame capture slots. */
 	for (unsigned int frame = 0; frame < 4; frame++) {
 		switch (pix->pixelformat) {
+		case V4L2_PIX_FMT_NV12M:
+			risp_core_write(core,
+					ISPCORE_DMA_IMAGE_FRAME_MODE(index, frame),
+					1);
+			risp_core_write(core,
+					ISPCORE_DMA_IMAGE_FRAME_PIXEL_POSITION(index, frame),
+					0 << 24 | 0 << 16 | 4 << 8 | 16 << 0);
+			risp_core_write(core,
+					ISPCORE_DMA_IMAGE_FRAME_PIXEL_BITWIDTH_MINUS1(index, frame),
+					0 << 24 | 0 << 16 | 7 << 8 | 7 << 0);
+			/*
+			 * Pick CrCb data from even lines.
+			 *
+			 * The datasheet suggest taking the odd lines but that
+			 * would require padding at the start of the buffer to
+			 * create valid NV12 frame, or start writing outside the
+			 * buffer potentially corrupting memory.
+			 */
+			risp_core_write(core,
+					ISPCORE_DMA_IMAGE_FRAME_PIXEL_BPP(index, frame),
+					0 << 28 | 0 << 24 |
+					0 << 20 | 0 << 16 |
+					1 << 12 | 0 << 8 |
+					3 << 4  | 0 << 0);
+
+			risp_core_write(core,
+					ISPCORE_DMA_IMAGE_FRAME_STRIDE_COMP0(index, frame),
+					pix->plane_fmt[0].bytesperline);
+			/*
+			 * Stride of CrCb plane needs be cut in half as the
+			 * DMA-engine writes blank pixels for the lines not
+			 * processed.
+			 *
+			 * See risp_io_capture_try_format() for details.
+			 */
+			risp_core_write(core,
+					ISPCORE_DMA_IMAGE_FRAME_STRIDE_COMP1(index, frame),
+					pix->plane_fmt[1].bytesperline / 2);
+			break;
 		case V4L2_PIX_FMT_NV16M:
 			risp_core_write(core,
 					ISPCORE_DMA_IMAGE_FRAME_MODE(index, frame),
@@ -436,6 +475,7 @@ static u32 risp_core_pix2bus(const struct rcar_isp_core_io *io)
 		return MEDIA_BUS_FMT_SRGGB12_1X12;
 	case V4L2_PIX_FMT_XBGR32:
 		return MEDIA_BUS_FMT_RGB888_1X24;
+	case V4L2_PIX_FMT_NV12M:
 	case V4L2_PIX_FMT_NV16M:
 		return MEDIA_BUS_FMT_YUYV12_1X24;
 	default:

-- 
2.55.0

Re: [PATCH 1/4] media: rcar-isp: ispcore: Honour requested bytesperline on RAW input

[Barnabás Pőcze]

Hi

2026. 10. 19. 21:35 keltezéssel, Niklas Söderlund írta:
> From: Yuya Hamamachi <yuya.hamamachi.sx@renesas.com>
> 
> The RAW input video device always overwrote the bytesperline requested
> by userspace with width * bpp. The input buffers are however shared with
> the VIN, which may use a larger line stride (for example 3296 instead of
> 3280 bytes for a 1640 pixel wide 10 bit image). The dmabuf exported by
> the ISP was then smaller than the VIN buffer size, and queueing it to
> the VIN failed with -EINVAL:
> 
>    __prepare_dmabuf: invalid dmabuf length 4042752 for plane 0,
>    minimum length 4060672
> 
> Use the requested bytesperline if it is larger than the minimum.
> 
> Signed-off-by: Yuya Hamamachi <yuya.hamamachi.sx@renesas.com>
> [Niklas: Use max(min(..)) instead of if statement]
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
>   drivers/media/platform/renesas/rcar-isp/core-io.c | 11 ++++++++++-
>   1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/renesas/rcar-isp/core-io.c b/drivers/media/platform/renesas/rcar-isp/core-io.c
> index b60b91f43d42..badde72352c1 100644
> --- a/drivers/media/platform/renesas/rcar-isp/core-io.c
> +++ b/drivers/media/platform/renesas/rcar-isp/core-io.c
> @@ -5,6 +5,8 @@
>    * Copyright (C) 2026 Ragnatech AB
>    */
>   
> +#include <linux/sizes.h>
> +
>   #include <media/v4l2-ctrls.h>
>   #include <media/v4l2-event.h>
>   #include <media/v4l2-ioctl.h>
> @@ -335,6 +337,7 @@ static void risp_io_input_try_format(struct rcar_isp_core_io *io,
>   				     struct v4l2_pix_format_mplane *pix)
>   {
>   	unsigned int bpp = 0;
> +	unsigned int bpl;
>   
>   	v4l_bound_align_image(&pix->width, RISP_MIN_WIDTH, RISP_MAX_WIDTH, RISP_SIZE_ALIGNMENT,
>   			      &pix->height, RISP_MIN_HEIGHT, RISP_MAX_HEIGHT, RISP_SIZE_ALIGNMENT,
> @@ -352,11 +355,17 @@ static void risp_io_input_try_format(struct rcar_isp_core_io *io,
>   		bpp = risp_io_input_formats[0].bpp;
>   	}
>   
> +	/*
> +	 * Honour the line stride requested by userspace, the buffers are
> +	 * shared with the VIN which may use a larger stride than the minimum.
> +	 */
> +	bpl = max(pix->width * bpp, min(pix->plane_fmt[0].bytesperline, SZ_64K));

   clamp(pix->plane_fmt[0].bytesperline, pix->width * bpp, SZ_64K)

?

And does the device really accept any value between these two limits?


> +
>   	pix->field = V4L2_FIELD_NONE;
>   	pix->colorspace = V4L2_COLORSPACE_RAW;
>   
>   	pix->num_planes = 1;
> -	pix->plane_fmt[0].bytesperline = pix->width * bpp;
> +	pix->plane_fmt[0].bytesperline = bpl;
>   	pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
>   }
>   
> 

Re: [PATCH 2/4] media: rcar-isp: ispcore: Update image data flush settings

[Barnabás Pőcze]

Hi

2026. 10. 19. 21:35 keltezéssel, Niklas Söderlund írta:
> Later versions of the datasheet have an updated chapter on how the image
> out flush settings depends on proper stride settings. Update the bytes
> per line calculation and flush out settings to match the datasheet
> clarification.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
>   drivers/media/platform/renesas/rcar-isp/core-io.c | 28 ++++++++++++++++++++---
>   drivers/media/platform/renesas/rcar-isp/core.c    |  2 +-
>   2 files changed, 26 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/rcar-isp/core-io.c b/drivers/media/platform/renesas/rcar-isp/core-io.c
> index badde72352c1..48db37bad69c 100644
> --- a/drivers/media/platform/renesas/rcar-isp/core-io.c
> +++ b/drivers/media/platform/renesas/rcar-isp/core-io.c
> @@ -733,6 +733,28 @@ static const struct v4l2_pix_format_mplane risp_io_capture_default_format = {
>   	},
>   };
>   
> +static unsigned int risp_io_calture_bytesperline(unsigned int width, unsigned int bpp)

   calture -> capture


> +{
> +	/*
> +	 * The datasheet specifies strict alignment and padding rules to be able
> +	 * to flush out all the pixels. Each line must be 256 byte aligned and
> +	 * at the end of the frame at least one extra line of space must be
> +	 * available.
> +	 *
> +	 * As we can't extend the buffer with one extra line we need to work
> +	 * with stride setting to satisfy both constraints. Fortunately the
> +	 * datasheet gives us a calculation for this.
> +	 *
> +	 *   FLUSH[pixels] = ALIGN(WIDTH[pixels], 256)
> +	 *
> +	 *   PADDING[bytes] = ALIGN(BPP[bytes] * WIDTH[pixels], 256) - BPP[bytes] * WIDTH[pixels] + BPP[bytes] * FLUSH[pixels]
> +	 */
> +	unsigned int data = bpp * width;
> +	unsigned int padding = ALIGN(data, 256) - data + bpp * ALIGN(width, 256);

So this adds one line's worth of extra padding on top of what is needed to achieve 256-byte alignment
of the actual data? With this change, for an output size of 2304x1296, I see

   ioctl(18</dev/video3>, VIDIOC_TRY_FMT, {
     type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, fmt.pix_mp={
       width=2304, height=1296,
       pixelformat=v4l2_fourcc('X', 'R', '2', '4') /* V4L2_PIX_FMT_XBGR32 */,
       field=V4L2_FIELD_NONE, colorspace=V4L2_COLORSPACE_DEFAULT,
       plane_fmt=[], num_planes=0
     }
   } => {
    fmt.pix_mp={
      width=2304, height=1296,
      pixelformat=v4l2_fourcc('X', 'R', '2', '4') /* V4L2_PIX_FMT_XBGR32 */,
      field=V4L2_FIELD_NONE,
      colorspace=V4L2_COLORSPACE_SRGB,
      plane_fmt=[
        { sizeimage=23887872, bytesperline=18432 }
      ],
      num_planes=1
    }
   }) = 0 <0.000074>

which means that the stride is `(2304 * 4) * 2 = 18432`, which is effectively
twice what would be required for just the 256 byte alignment. So this
in practice doubles the size of every buffer, or am I missing something?


> +
> +	return data + padding;
> +}
> +
>   static void risp_io_capture_try_format(struct rcar_isp_core_io *io,
>   				       struct v4l2_pix_format_mplane *pix)
>   {
> @@ -749,16 +771,16 @@ static void risp_io_capture_try_format(struct rcar_isp_core_io *io,
>   	case V4L2_PIX_FMT_NV16M:
>   		pix->quantization = V4L2_QUANTIZATION_LIM_RANGE;
>   		pix->num_planes = 2;
> -		pix->plane_fmt[0].bytesperline = ALIGN(pix->width, 256);
> +		pix->plane_fmt[0].bytesperline = risp_io_calture_bytesperline(pix->width, 1);
>   		pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
> -		pix->plane_fmt[1].bytesperline = ALIGN(pix->width, 256);
> +		pix->plane_fmt[1].bytesperline = pix->plane_fmt[0].bytesperline;
>   		pix->plane_fmt[1].sizeimage = pix->plane_fmt[1].bytesperline * pix->height;
>   		break;
>   	default:
>   		pix->pixelformat = V4L2_PIX_FMT_XBGR32;
>   		pix->quantization = V4L2_QUANTIZATION_FULL_RANGE;
>   		pix->num_planes = 1;
> -		pix->plane_fmt[0].bytesperline = ALIGN(pix->width * 4, 256);
> +		pix->plane_fmt[0].bytesperline = risp_io_calture_bytesperline(pix->width, 4);
>   		pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
>   		break;
>   	}
> diff --git a/drivers/media/platform/renesas/rcar-isp/core.c b/drivers/media/platform/renesas/rcar-isp/core.c
> index b5861d0cd0e8..3e135e84b75b 100644
> --- a/drivers/media/platform/renesas/rcar-isp/core.c
> +++ b/drivers/media/platform/renesas/rcar-isp/core.c
> @@ -398,7 +398,7 @@ static int risp_core_config_output(struct rcar_isp_core *core,
>   
>   	/* Set image out flush EOF. */
>   	risp_core_write(core, ISPCORE_DMA_IMAGE_FLUSH_OUT_REG(index),
> -			pix->plane_fmt[0].bytesperline <<
> +			ALIGN(pix->width, 256) <<

I'm probably mistaken, but does this not need a multiplication by bpp?
Or was the previous version actually incorrect?


>   			ISPCORE_DMA_IMAGE_FLUSH_OUT_PADDING_PIXEL_EOF_SHIFT);
>   
>   	/* Enable DMA and set burst length. */
> 

Re: [PATCH 3/4] media: rcar-isp: ispcore: Look at number of planes when programming

[Barnabás Pőcze]

2026. 10. 19. 21:35 keltezéssel, Niklas Söderlund írta:
> Instead of expanding a list of formats that uses two planes look at the
> format information directly.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
>   drivers/media/platform/renesas/rcar-isp/core.c | 3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/rcar-isp/core.c b/drivers/media/platform/renesas/rcar-isp/core.c
> index 3e135e84b75b..cb9d9033d0cf 100644
> --- a/drivers/media/platform/renesas/rcar-isp/core.c
> +++ b/drivers/media/platform/renesas/rcar-isp/core.c
> @@ -126,8 +126,7 @@ static void risp_core_job_run_output(struct rcar_isp_core *core,
>   		mem = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
>   		risp_core_write(core, reg, mem);
>   
> -		/* Only NV16 uses 2 planes. */
> -		if (fmt->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_NV16M)
> +		if (fmt->fmt.pix_mp.num_planes != 2)
>   			continue;

Looks reasonable to me. Maybe `< 2` would be more future proof?

Reviewed-by: Barnabás Pőcze <barnabas.pocze+renesas@ideasonboard.com>


>   
>   		reg = ISPCORE_DMA_IMAGE_FRAME_BASE_ADDRESS_COMP1(0, frame);
> 

Re: [PATCH 2/4] media: rcar-isp: ispcore: Update image data flush settings

[Niklas Söderlund]

Hi Barnabás,

Thanks for your feedback.

On 2026-10-20 12:19:54 +0200, Barnabás Pőcze wrote:
> Hi
> 
> 2026. 10. 19. 21:35 keltezéssel, Niklas Söderlund írta:
> > Later versions of the datasheet have an updated chapter on how the image
> > out flush settings depends on proper stride settings. Update the bytes
> > per line calculation and flush out settings to match the datasheet
> > clarification.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > ---
> >   drivers/media/platform/renesas/rcar-isp/core-io.c | 28 ++++++++++++++++++++---
> >   drivers/media/platform/renesas/rcar-isp/core.c    |  2 +-
> >   2 files changed, 26 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/media/platform/renesas/rcar-isp/core-io.c b/drivers/media/platform/renesas/rcar-isp/core-io.c
> > index badde72352c1..48db37bad69c 100644
> > --- a/drivers/media/platform/renesas/rcar-isp/core-io.c
> > +++ b/drivers/media/platform/renesas/rcar-isp/core-io.c
> > @@ -733,6 +733,28 @@ static const struct v4l2_pix_format_mplane risp_io_capture_default_format = {
> >   	},
> >   };
> > +static unsigned int risp_io_calture_bytesperline(unsigned int width, unsigned int bpp)
> 
>   calture -> capture
> 
> 
> > +{
> > +	/*
> > +	 * The datasheet specifies strict alignment and padding rules to be able
> > +	 * to flush out all the pixels. Each line must be 256 byte aligned and
> > +	 * at the end of the frame at least one extra line of space must be
> > +	 * available.
> > +	 *
> > +	 * As we can't extend the buffer with one extra line we need to work
> > +	 * with stride setting to satisfy both constraints. Fortunately the
> > +	 * datasheet gives us a calculation for this.
> > +	 *
> > +	 *   FLUSH[pixels] = ALIGN(WIDTH[pixels], 256)
> > +	 *
> > +	 *   PADDING[bytes] = ALIGN(BPP[bytes] * WIDTH[pixels], 256) - BPP[bytes] * WIDTH[pixels] + BPP[bytes] * FLUSH[pixels]
> > +	 */
> > +	unsigned int data = bpp * width;
> > +	unsigned int padding = ALIGN(data, 256) - data + bpp * ALIGN(width, 256);
> 
> So this adds one line's worth of extra padding on top of what is needed to achieve 256-byte alignment
> of the actual data? With this change, for an output size of 2304x1296, I see
> 
>   ioctl(18</dev/video3>, VIDIOC_TRY_FMT, {
>     type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, fmt.pix_mp={
>       width=2304, height=1296,
>       pixelformat=v4l2_fourcc('X', 'R', '2', '4') /* V4L2_PIX_FMT_XBGR32 */,
>       field=V4L2_FIELD_NONE, colorspace=V4L2_COLORSPACE_DEFAULT,
>       plane_fmt=[], num_planes=0
>     }
>   } => {
>    fmt.pix_mp={
>      width=2304, height=1296,
>      pixelformat=v4l2_fourcc('X', 'R', '2', '4') /* V4L2_PIX_FMT_XBGR32 */,
>      field=V4L2_FIELD_NONE,
>      colorspace=V4L2_COLORSPACE_SRGB,
>      plane_fmt=[
>        { sizeimage=23887872, bytesperline=18432 }
>      ],
>      num_planes=1
>    }
>   }) = 0 <0.000074>
> 
> which means that the stride is `(2304 * 4) * 2 = 18432`, which is effectively
> twice what would be required for just the 256 byte alignment. So this
> in practice doubles the size of every buffer, or am I missing something?

No that is correct, check chapter 54.5.11 Notes of additional memory 
area of flush out (Image Output DMA).

I'm quiet sure we could be more clever add add the flush out after the 
image by sizeimage = bpp * (hight + 2) or something like that. I opted 
for the big hammer here as this large padding space is needed to support 
NV12 due to how the DMA-engine operates.

In future we could look at using this more clever way for NV16 and RGB 
which do not need the large padding are to operate properly.

> 
> 
> > +
> > +	return data + padding;
> > +}
> > +
> >   static void risp_io_capture_try_format(struct rcar_isp_core_io *io,
> >   				       struct v4l2_pix_format_mplane *pix)
> >   {
> > @@ -749,16 +771,16 @@ static void risp_io_capture_try_format(struct rcar_isp_core_io *io,
> >   	case V4L2_PIX_FMT_NV16M:
> >   		pix->quantization = V4L2_QUANTIZATION_LIM_RANGE;
> >   		pix->num_planes = 2;
> > -		pix->plane_fmt[0].bytesperline = ALIGN(pix->width, 256);
> > +		pix->plane_fmt[0].bytesperline = risp_io_calture_bytesperline(pix->width, 1);
> >   		pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
> > -		pix->plane_fmt[1].bytesperline = ALIGN(pix->width, 256);
> > +		pix->plane_fmt[1].bytesperline = pix->plane_fmt[0].bytesperline;
> >   		pix->plane_fmt[1].sizeimage = pix->plane_fmt[1].bytesperline * pix->height;
> >   		break;
> >   	default:
> >   		pix->pixelformat = V4L2_PIX_FMT_XBGR32;
> >   		pix->quantization = V4L2_QUANTIZATION_FULL_RANGE;
> >   		pix->num_planes = 1;
> > -		pix->plane_fmt[0].bytesperline = ALIGN(pix->width * 4, 256);
> > +		pix->plane_fmt[0].bytesperline = risp_io_calture_bytesperline(pix->width, 4);
> >   		pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
> >   		break;
> >   	}
> > diff --git a/drivers/media/platform/renesas/rcar-isp/core.c b/drivers/media/platform/renesas/rcar-isp/core.c
> > index b5861d0cd0e8..3e135e84b75b 100644
> > --- a/drivers/media/platform/renesas/rcar-isp/core.c
> > +++ b/drivers/media/platform/renesas/rcar-isp/core.c
> > @@ -398,7 +398,7 @@ static int risp_core_config_output(struct rcar_isp_core *core,
> >   	/* Set image out flush EOF. */
> >   	risp_core_write(core, ISPCORE_DMA_IMAGE_FLUSH_OUT_REG(index),
> > -			pix->plane_fmt[0].bytesperline <<
> > +			ALIGN(pix->width, 256) <<
> 
> I'm probably mistaken, but does this not need a multiplication by bpp?
> Or was the previous version actually incorrect?

The whole DMA Flush operation was a tad missunderstood but is better 
documented now. The unit here is pixels not bytes. This register covers 
all component outputs and each component may have a different bpp.

> 
> 
> >   			ISPCORE_DMA_IMAGE_FLUSH_OUT_PADDING_PIXEL_EOF_SHIFT);
> >   	/* Enable DMA and set burst length. */
> > 
> 

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH 1/4] media: rcar-isp: ispcore: Honour requested bytesperline on RAW input

[Niklas Söderlund]

Hi Barnabás,

Thanks for your review.

On 2026-10-20 11:36:04 +0200, Barnabás Pőcze wrote:
> Hi
> 
> 2026. 10. 19. 21:35 keltezéssel, Niklas Söderlund írta:
> > From: Yuya Hamamachi <yuya.hamamachi.sx@renesas.com>
> > 
> > The RAW input video device always overwrote the bytesperline requested
> > by userspace with width * bpp. The input buffers are however shared with
> > the VIN, which may use a larger line stride (for example 3296 instead of
> > 3280 bytes for a 1640 pixel wide 10 bit image). The dmabuf exported by
> > the ISP was then smaller than the VIN buffer size, and queueing it to
> > the VIN failed with -EINVAL:
> > 
> >    __prepare_dmabuf: invalid dmabuf length 4042752 for plane 0,
> >    minimum length 4060672
> > 
> > Use the requested bytesperline if it is larger than the minimum.
> > 
> > Signed-off-by: Yuya Hamamachi <yuya.hamamachi.sx@renesas.com>
> > [Niklas: Use max(min(..)) instead of if statement]
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > ---
> >   drivers/media/platform/renesas/rcar-isp/core-io.c | 11 ++++++++++-
> >   1 file changed, 10 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/platform/renesas/rcar-isp/core-io.c b/drivers/media/platform/renesas/rcar-isp/core-io.c
> > index b60b91f43d42..badde72352c1 100644
> > --- a/drivers/media/platform/renesas/rcar-isp/core-io.c
> > +++ b/drivers/media/platform/renesas/rcar-isp/core-io.c
> > @@ -5,6 +5,8 @@
> >    * Copyright (C) 2026 Ragnatech AB
> >    */
> > +#include <linux/sizes.h>
> > +
> >   #include <media/v4l2-ctrls.h>
> >   #include <media/v4l2-event.h>
> >   #include <media/v4l2-ioctl.h>
> > @@ -335,6 +337,7 @@ static void risp_io_input_try_format(struct rcar_isp_core_io *io,
> >   				     struct v4l2_pix_format_mplane *pix)
> >   {
> >   	unsigned int bpp = 0;
> > +	unsigned int bpl;
> >   	v4l_bound_align_image(&pix->width, RISP_MIN_WIDTH, RISP_MAX_WIDTH, RISP_SIZE_ALIGNMENT,
> >   			      &pix->height, RISP_MIN_HEIGHT, RISP_MAX_HEIGHT, RISP_SIZE_ALIGNMENT,
> > @@ -352,11 +355,17 @@ static void risp_io_input_try_format(struct rcar_isp_core_io *io,
> >   		bpp = risp_io_input_formats[0].bpp;
> >   	}
> > +	/*
> > +	 * Honour the line stride requested by userspace, the buffers are
> > +	 * shared with the VIN which may use a larger stride than the minimum.
> > +	 */
> > +	bpl = max(pix->width * bpp, min(pix->plane_fmt[0].bytesperline, SZ_64K));
> 
>   clamp(pix->plane_fmt[0].bytesperline, pix->width * bpp, SZ_64K)

Good idea.

> 
> ?
> 
> And does the device really accept any value between these two limits?

The stride register where this is programmed is 16 bit wide.

> 
> 
> > +
> >   	pix->field = V4L2_FIELD_NONE;
> >   	pix->colorspace = V4L2_COLORSPACE_RAW;
> >   	pix->num_planes = 1;
> > -	pix->plane_fmt[0].bytesperline = pix->width * bpp;
> > +	pix->plane_fmt[0].bytesperline = bpl;
> >   	pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
> >   }
> > 
> 

-- 
Kind Regards,
Niklas Söderlund