mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
To: "Niklas Söderlund" <niklas.soderlund+renesas@ragnatech.se>
Cc: "Jacopo Mondi" <jacopo.mondi+renesas@ideasonboard.com>,
	"Jai Luthra" <jai.luthra+renesas@ideasonboard.com>,
	"Niklas Söderlund" <niklas.soderlund@ragnatech.se>,
	"Mauro Carvalho Chehab" <mchehab@kernel.org>,
	"Geert Uytterhoeven" <geert+renesas@glider.be>,
	"Magnus Damm" <magnus.damm@gmail.com>,
	"Barnabás Pőcze" <barnabas.pocze+renesas@ideasonboard.com>,
	"Yusuke Goda" <yusuke.goda.sx@renesas.com>,
	"Yuya Hamamachi" <yuya.hamamachi.sx@renesas.com>,
	linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-renesas-soc@vger.kernel.org
Subject: Re: [PATCH v2 4/4] media: rcar-isp: ispcore: Add NV12M support
Date: Tue, 17 Nov 2026 15:39:57 +0100	[thread overview]
Message-ID: <avxfncW0vkr01pWp@zed> (raw)
In-Reply-To: <20261114-v4l2-add-nv12-v2-4-a3f5ecfd48d6@ragnatech.se>

Hi Niklas

On Sat, Nov 14, 2026 at 08:55:19PM +0100, Niklas Söderlund wrote:
> It is possible to support NV12M output by only outputting the CbCr plane
> from even or odd lines. Enabling support for it is straightforward but
> one snag around how the DMA-engine works.
>
> When the DMA-engine is configured to only output CbCr data from odd or
> even lines it still outputs blanks pixels to the buffer for the line
> where no data is taken.
>
> To work around this and still produce valid NV12 frames the device needs
> to be configured so all blank pixels written to the buffer are part of
> the line padding. In addition to also keeping the same number of padding
> bytes as the Y plane.

To confirm my understanding:

The Y and CbCr planes have the same strides because:

1) in 2/4 the line stride gets doubled for the Y plane and the stride
is copied to the CbCr plane
2) the CbCr plane needs a doubled line stride  because empty data will
end up being written to the second half of each line

Is this correct ?

>
> The DMA-engine is able to do this as we can set different strides for
> the two planes and haw two DMA-engine lines (one with data + some
> padding and one with only padding) make up a single correctly formatted
> NV12 CbCr line.
>
> For readers comparing the DMA configuration with the datasheet this
> driver takes the CbCr data from even lines, compared to odd lines in the
> datasheet example. The reason for this is that if odd lines are used the
> first even line adds half a NV12 line of padding at the start of the
> CbCr plane. To counter this the CbCr plane would need a padding line at
> the start of the plane which is not supported, or we would need to set
> the plane start address half a line before the buffer actually starts
> and potentially corrupt memory as we write blank bytes outside our
> buffer.

Have you considered the following section of the chip TRM:

(3). Memory image on external memory in YUV420/LUV420 format.

Reading it suggests that the DENSE_LINE_MODE bit of register
DMA_IMAGE_AXI_CONFIG can be used to avoid outputing empty lines when
using YUV420 subsampling.

The documentation says the STRIDE value is halved from what is written
into the register when DENSE mode is used. So you would probably need
to write a 2*stride in the register but you don't actually need
to report a doubled bytesperline in 'struct v4l2_pix_format_mplane`
from risp_io_input_try_format() ?

>
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
>  drivers/media/platform/renesas/rcar-isp/core-io.c | 39 +++++++++++++++++++++-
>  drivers/media/platform/renesas/rcar-isp/core.c    | 40 +++++++++++++++++++++++
>  2 files changed, 78 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/renesas/rcar-isp/core-io.c b/drivers/media/platform/renesas/rcar-isp/core-io.c
> index 1c63194f6455..d3b302c5b66e 100644
> --- a/drivers/media/platform/renesas/rcar-isp/core-io.c
> +++ b/drivers/media/platform/renesas/rcar-isp/core-io.c
> @@ -768,6 +768,39 @@ static void risp_io_capture_try_format(struct rcar_isp_core_io *io,
>  	pix->xfer_func = V4L2_XFER_FUNC_SRGB;
>
>  	switch (pix->pixelformat) {
> +	case V4L2_PIX_FMT_NV12M:
> +		pix->quantization = V4L2_QUANTIZATION_LIM_RANGE;
> +		pix->num_planes = 2;
> +		/*
> +		 * The DMA-engine still outputs pixels for the height lines of
> +		 * the CrCb plane. To produce a NV12 frame we must ensure all
> +		 * these blank pixels are in the padding area of the frame.
> +		 *
> +		 * To do this the stride of the CrCb plane need to be half that
> +		 * of the Y plane. The first "DMA-engine line" will write the
> +		 * CrCb data + some blank padding, while the second "DMA-engine
> +		 * line" will only write blank padding bytes. Two "DMA-engine
> +		 * lines" will then create one correct NV12 line in the CrCb
> +		 * plane.
> +		 *
> +		 * All frame data also needs to fits in the first half of the
> +		 * line. The strict DMA flush settings used already ensures
> +		 * this is always the case.
> +		 *
> +		 * Finally the NV12 stride of the Y and CrCb plane must be the
> +		 * same and the R-Car ISP needs the hardware stride to be 256
> +		 * byte aligned. By align the Y plane lines to 512 bytes the
> +		 * CrCb planes hardware stride is always aligned with 256 when
> +		 * cut in half.
> +		 */
> +		pix->plane_fmt[0].bytesperline =
> +			ALIGN(risp_io_capture_bytesperline(pix->width, 1), 512);
> +		pix->plane_fmt[0].sizeimage = pix->plane_fmt[0].bytesperline * pix->height;
> +		/* Both planes must have the same length. */
> +		pix->plane_fmt[1].bytesperline = pix->plane_fmt[0].bytesperline;
> +		/* The CrCb plane is half height of the Y plane. */
> +		pix->plane_fmt[1].sizeimage = pix->plane_fmt[1].bytesperline * pix->height / 2;
> +		break;
>  	case V4L2_PIX_FMT_NV16M:
>  		pix->quantization = V4L2_QUANTIZATION_LIM_RANGE;
>  		pix->num_planes = 2;
> @@ -794,9 +827,12 @@ static int risp_io_capture_enum_fmt(struct file *file, void *priv,
>
>  	switch (f->index) {
>  	case 0:
> -		f->pixelformat = V4L2_PIX_FMT_NV16M;
> +		f->pixelformat = V4L2_PIX_FMT_NV12M;
>  		break;
>  	case 1:
> +		f->pixelformat = V4L2_PIX_FMT_NV16M;
> +		break;
> +	case 2:
>  		f->pixelformat = V4L2_PIX_FMT_XBGR32;
>  		break;
>  	default:
> @@ -854,6 +890,7 @@ static int risp_io_capture_enum_framesizes(struct file *file, void *fh,
>  		return -EINVAL;
>
>  	switch (fsize->pixel_format) {
> +	case V4L2_PIX_FMT_NV12M:
>  	case V4L2_PIX_FMT_NV16M:
>  	case V4L2_PIX_FMT_XBGR32:
>  		break;
> diff --git a/drivers/media/platform/renesas/rcar-isp/core.c b/drivers/media/platform/renesas/rcar-isp/core.c
> index 5cd21134a2d7..76be90e78420 100644
> --- a/drivers/media/platform/renesas/rcar-isp/core.c
> +++ b/drivers/media/platform/renesas/rcar-isp/core.c
> @@ -341,6 +341,45 @@ static int risp_core_config_output(struct rcar_isp_core *core,
>  	/* For all frame capture slots. */
>  	for (unsigned int frame = 0; frame < 4; frame++) {
>  		switch (pix->pixelformat) {
> +		case V4L2_PIX_FMT_NV12M:
> +			risp_core_write(core,
> +					ISPCORE_DMA_IMAGE_FRAME_MODE(index, frame),
> +					1);
> +			risp_core_write(core,
> +					ISPCORE_DMA_IMAGE_FRAME_PIXEL_POSITION(index, frame),
> +					0 << 24 | 0 << 16 | 4 << 8 | 16 << 0);
> +			risp_core_write(core,
> +					ISPCORE_DMA_IMAGE_FRAME_PIXEL_BITWIDTH_MINUS1(index, frame),
> +					0 << 24 | 0 << 16 | 7 << 8 | 7 << 0);
> +			/*
> +			 * Pick CrCb data from even lines.
> +			 *
> +			 * The datasheet suggest taking the odd lines but that
> +			 * would require padding at the start of the buffer to
> +			 * create valid NV12 frame, or start writing outside the
> +			 * buffer potentially corrupting memory.
> +			 */
> +			risp_core_write(core,
> +					ISPCORE_DMA_IMAGE_FRAME_PIXEL_BPP(index, frame),
> +					0 << 28 | 0 << 24 |
> +					0 << 20 | 0 << 16 |
> +					1 << 12 | 0 << 8 |
> +					3 << 4  | 0 << 0);
> +
> +			risp_core_write(core,
> +					ISPCORE_DMA_IMAGE_FRAME_STRIDE_COMP0(index, frame),
> +					pix->plane_fmt[0].bytesperline);
> +			/*
> +			 * Stride of CrCb plane needs be cut in half as the
> +			 * DMA-engine writes blank pixels for the lines not
> +			 * processed.
> +			 *
> +			 * See risp_io_capture_try_format() for details.
> +			 */
> +			risp_core_write(core,
> +					ISPCORE_DMA_IMAGE_FRAME_STRIDE_COMP1(index, frame),
> +					pix->plane_fmt[1].bytesperline / 2);

Ah here it is halved!

We double the Y and CbCr planes stride (in 2/4) to accommodate the
blank pixels (when using !DENSE mode), and here we halve it ? Is this
because with 420 subsampling the CbCr plane is half the height of the
Y plane ? So why doubling the value of pix->plane_fmt[1].bytesperline
and then halve it when writing it to registers ? To over-allocate
half a plane and fill it with blank lines ?

I'm a bit surprised that !DENSE is the default mode, as it effecively
requires to allocate 2 identically-sized planes for Y and CbCr,
effectivelly filling half of the CbCr plane with non valid data.

Do you end up with interleaved valid/non-valid lines in the CbCr plane ?
How do these get discarded when the image is interpreted by userspace ?

> +			break;
>  		case V4L2_PIX_FMT_NV16M:
>  			risp_core_write(core,
>  					ISPCORE_DMA_IMAGE_FRAME_MODE(index, frame),
> @@ -436,6 +475,7 @@ static u32 risp_core_pix2bus(const struct rcar_isp_core_io *io)
>  		return MEDIA_BUS_FMT_SRGGB12_1X12;
>  	case V4L2_PIX_FMT_XBGR32:
>  		return MEDIA_BUS_FMT_RGB888_1X24;
> +	case V4L2_PIX_FMT_NV12M:
>  	case V4L2_PIX_FMT_NV16M:
>  		return MEDIA_BUS_FMT_YUYV12_1X24;
>  	default:
>
> --
> 2.55.0
>
>

      reply	other threads:[~2026-11-17 14:40 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-14 19:55 [PATCH v2 0/4] media: rcar-isp: Add NV12M support and fix stride Niklas Söderlund
2026-11-14 19:55 ` [PATCH v2 1/4] media: rcar-isp: ispcore: Honour requested bytesperline on RAW input Niklas Söderlund
2026-11-16 11:21   ` Jacopo Mondi
2026-11-14 19:55 ` [PATCH v2 2/4] media: rcar-isp: ispcore: Update image data flush settings Niklas Söderlund
2026-11-16 12:21   ` Jacopo Mondi
2026-11-14 19:55 ` [PATCH v2 3/4] media: rcar-isp: ispcore: Look at number of planes when programming Niklas Söderlund
2026-11-14 19:55 ` [PATCH v2 4/4] media: rcar-isp: ispcore: Add NV12M support Niklas Söderlund
2026-11-17 14:39   ` Jacopo Mondi [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=avxfncW0vkr01pWp@zed \
    --to=jacopo.mondi@ideasonboard.com \
    --cc=barnabas.pocze+renesas@ideasonboard.com \
    --cc=geert+renesas@glider.be \
    --cc=jacopo.mondi+renesas@ideasonboard.com \
    --cc=jai.luthra+renesas@ideasonboard.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-renesas-soc@vger.kernel.org \
    --cc=magnus.damm@gmail.com \
    --cc=mchehab@kernel.org \
    --cc=niklas.soderlund+renesas@ragnatech.se \
    --cc=niklas.soderlund@ragnatech.se \
    --cc=yusuke.goda.sx@renesas.com \
    --cc=yuya.hamamachi.sx@renesas.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®