mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Philipp Zabel <p.zabel@pengutronix.de>
To: Sascha Hauer <s.hauer@pengutronix.de>,
	Lucas Sinn	 <lucas.sinn@wolfvision.net>,
	Mauro Carvalho Chehab <mchehab@kernel.org>,
	Rob Herring <robh@kernel.org>,
	Krzysztof Kozlowski <krzk+dt@kernel.org>,
	Conor Dooley	 <conor+dt@kernel.org>,
	Heiko Stuebner <heiko@sntech.de>
Cc: linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org,
	 devicetree@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	 linux-kernel@vger.kernel.org
Subject: Re: [PATCH v7 2/4] media: rockchip: Add JPEG decoder driver
Date: Fri, 20 Nov 2026 15:58:19 +0100	[thread overview]
Message-ID: <266c9fe4a9aca943c2c2be39a7dc0f3bea2df362.camel@pengutronix.de> (raw)
In-Reply-To: <20261022-rockchip-jpegdec-v7-2-3fd7dd55f408@pengutronix.de>

On Do, 2026-10-22 at 13:06 +0200, Sascha Hauer wrote:
> Add a driver for the JPEG hardware decoder Rockchip integrates into a
> number of its SoCs.  Downstream it is known as the VDPU720, which is the
> name the register definitions carry here.
> 
> Exposes one V4L2 M2M device implementing the stateful decoder interface:
> JPEG input -> NV12 output, or GREY for a grayscale frame.  The frame
> header is parsed on the CPU when a coded buffer is queued.  The first
> buffer queued while the capture queue is stopped sets the capture format
> and raises V4L2_EVENT_SOURCE_CHANGE, so userspace learns the frame size
> without parsing the bitstream itself.  V4L2_DEC_CMD_STOP drains.
> 
> Dynamic resolution change is supported, and has to be for GStreamer:
> without V4L2_FMT_FLAG_DYN_RESOLUTION its v4l2videodec does not wait for
> the initial source change and later takes the pending event for one in
> the middle of the stream, with the flag it relies on the driver to
> follow a change.  A frame that decodes to another size or pixel format
> than the one queued before it is marked when it is queued.  The decoder
> stops in front of it, reports the change, returns an empty LAST capture
> buffer and resumes once userspace has restarted the capture queue or
> sent V4L2_DEC_CMD_START.  The capture format is only ever changed from
> an ioctl while no job can run.
> 
> Hardware requirements:
> - a DMA side buffer with Q-tables (zigzag->raster), Huffman mincode and
>   value tables, rebuilt from the frame header on every run
> - a two-phase IRQ clear sequence, which some revisions require.  It is
>   done unconditionally, see rkjpegd-vdpu720-regs.h
> - 16-byte stream alignment with a start-byte offset
> - MCU-aligned PIC_H (e.g. 1080p YUV420 needs 1088, not 1080)
> - FILL_DOWN_E on every NV12 conversion, the output chroma is vertically
>   subsampled so the hardware completes the bottom of the picture
> - DRI (restart interval) support when present
> 
> Every address register is a single 32 bit one with no high order
> companion anywhere in the map, so the DMA masks are capped accordingly.
> That is no constraint in practice: the block sits behind the Rockchip
> IOMMU, whose address space is 32 bit wide by construction, and memory
> above 4GB is reached through it.
> 
> A coded buffer that ends before the frame does is left to the hardware:
> BUF_EMPTY is enabled for every job and fails it.  A capture buffer
> smaller than the negotiated format fails the job too.  A failed frame
> comes back with the ERROR flag and its full payload.  The CPU never
> touches a capture buffer, so that queue has no kernel mapping.
> 
> A job that ends in error, or leaves the block unparked, is followed by an
> in-block soft reset, and so is a job the watchdog has to end.  The
> interrupt handler decodes the error status registers into a readable
> diagnostic.
> 
> Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
> ---
>  MAINTAINERS                                        |    1 +
>  drivers/media/platform/rockchip/Kconfig            |    1 +
>  drivers/media/platform/rockchip/Makefile           |    1 +
>  drivers/media/platform/rockchip/rkjpegd/Kconfig    |   16 +
>  drivers/media/platform/rockchip/rkjpegd/Makefile   |    4 +
>  .../rockchip/rkjpegd/rkjpegd-vdpu720-regs.h        |  235 +++
>  drivers/media/platform/rockchip/rkjpegd/rkjpegd.c  | 2222 ++++++++++++++++++++
>  7 files changed, 2480 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index f47f614dc6857..26a5101a6d76d 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -23721,6 +23721,7 @@ L:	linux-media@vger.kernel.org
>  L:	linux-rockchip@lists.infradead.org
>  S:	Maintained
>  F:	Documentation/devicetree/bindings/media/rockchip,rk3568-jpegd.yaml
> +F:	drivers/media/platform/rockchip/rkjpegd/
>  
>  ROCKCHIP RK3568 RANDOM NUMBER GENERATOR SUPPORT
>  M:	Daniel Golle <daniel@makrotopia.org>
> diff --git a/drivers/media/platform/rockchip/Kconfig b/drivers/media/platform/rockchip/Kconfig
> index ba401d32f01ba..53d28eb89e472 100644
> --- a/drivers/media/platform/rockchip/Kconfig
> +++ b/drivers/media/platform/rockchip/Kconfig
> @@ -5,4 +5,5 @@ comment "Rockchip media platform drivers"
>  source "drivers/media/platform/rockchip/rga/Kconfig"
>  source "drivers/media/platform/rockchip/rkcif/Kconfig"
>  source "drivers/media/platform/rockchip/rkisp1/Kconfig"
> +source "drivers/media/platform/rockchip/rkjpegd/Kconfig"
>  source "drivers/media/platform/rockchip/rkvdec/Kconfig"
> diff --git a/drivers/media/platform/rockchip/Makefile b/drivers/media/platform/rockchip/Makefile
> index 0e0b2cbbd4bd2..c256c51ced269 100644
> --- a/drivers/media/platform/rockchip/Makefile
> +++ b/drivers/media/platform/rockchip/Makefile
> @@ -2,4 +2,5 @@
>  obj-y += rga/
>  obj-y += rkcif/
>  obj-y += rkisp1/
> +obj-y += rkjpegd/
>  obj-y += rkvdec/
> diff --git a/drivers/media/platform/rockchip/rkjpegd/Kconfig b/drivers/media/platform/rockchip/rkjpegd/Kconfig
> new file mode 100644
> index 0000000000000..4c269d4a6654e
> --- /dev/null
> +++ b/drivers/media/platform/rockchip/rkjpegd/Kconfig
> @@ -0,0 +1,16 @@
> +# SPDX-License-Identifier: GPL-2.0
> +config VIDEO_ROCKCHIP_JPEGD
> +	tristate "Rockchip JPEG decoder driver"
> +	depends on V4L_MEM2MEM_DRIVERS
> +	depends on ARCH_ROCKCHIP || COMPILE_TEST
> +	depends on VIDEO_DEV
> +	depends on PM
> +	select V4L2_JPEG_HELPER
> +	select V4L2_MEM2MEM_DEV
> +	select VIDEOBUF2_DMA_CONTIG
> +	help
> +	  Support for the VDPU720 JPEG decoder found on Rockchip SoCs such
> +	  as the RK3568 and RK3588, decoding JPEG and MJPEG frames to NV12,
> +	  or to GREY for grayscale frames.
> +	  To compile this driver as a module, choose M here: the module
> +	  will be called rockchip-jpegd.
> diff --git a/drivers/media/platform/rockchip/rkjpegd/Makefile b/drivers/media/platform/rockchip/rkjpegd/Makefile
> new file mode 100644
> index 0000000000000..0acf2e6ac6785
> --- /dev/null
> +++ b/drivers/media/platform/rockchip/rkjpegd/Makefile
> @@ -0,0 +1,4 @@
> +# SPDX-License-Identifier: GPL-2.0
> +obj-$(CONFIG_VIDEO_ROCKCHIP_JPEGD) += rockchip-jpegd.o
> +
> +rockchip-jpegd-y += rkjpegd.o
> diff --git a/drivers/media/platform/rockchip/rkjpegd/rkjpegd-vdpu720-regs.h b/drivers/media/platform/rockchip/rkjpegd/rkjpegd-vdpu720-regs.h
> new file mode 100644
> index 0000000000000..eca01174804d6
> --- /dev/null
> +++ b/drivers/media/platform/rockchip/rkjpegd/rkjpegd-vdpu720-regs.h
> @@ -0,0 +1,235 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Rockchip VPU720 JPEG decoder register definitions
> + *
> + * Derived from downstream Rockchip MPP HAL (hal_jpegd_rkv_reg.h).
> + * Copyright (C) 2020 Rockchip Electronics Co., Ltd.
> + * Copyright (C) 2026 WolfVision GmbH
> + */
> +#ifndef RKJPEGD_VDPU720_REGS_H_
> +#define RKJPEGD_VDPU720_REGS_H_
> +
> +#include <linux/bitfield.h>
> +#include <linux/bits.h>
> +#include <linux/align.h>
> +#include <linux/types.h>
> +
> +#define VDPU720_REG_VERSION		0x000
> +#define VDPU720_PROD_NUM		GENMASK(31, 16)
> +#define VDPU720_BIT_DEPTH		BIT(8)
> +
> +#define VDPU720_REG_INT			0x004
> +#define VDPU720_DEC_E			BIT(0)
> +#define VDPU720_IRQ_DIS			BIT(1)
> +#define VDPU720_TIMEOUT_E		BIT(2)
> +#define VDPU720_BUF_EMPTY_E		BIT(3)
> +#define VDPU720_BUF_EMPTY_RELOAD	BIT(4)
> +#define VDPU720_SOFT_RST_EN		BIT(5)
> +#define VDPU720_IRQ_RAW			BIT(6)
> +#define VDPU720_WAIT_RESET_E		BIT(7)
> +#define VDPU720_IRQ			BIT(8)
> +#define VDPU720_DEC_RDY			BIT(9)
> +#define VDPU720_BUS_ERR			BIT(10)
> +#define VDPU720_DEC_ERR			BIT(11)
> +#define VDPU720_TIMEOUT			BIT(12)
> +#define VDPU720_BUF_EMPTY		BIT(13)
> +#define VDPU720_SOFT_RST_RDY		BIT(14)
> +
> +/* Error status bits used to decide whether a hardware reset is needed */
> +#define VDPU720_ERR_MASK		(VDPU720_BUS_ERR | VDPU720_DEC_ERR | \
> +					 VDPU720_TIMEOUT | VDPU720_BUF_EMPTY)
> +
> +/*
> + * VPU720 IRQ clear mask.
> + *
> + * Some revisions require a two-step IRQ acknowledgment before checking
> + * VDPU720_IRQ_RAW: write the status back with only the enable and control
> + * bits kept, then write 0 to fully clear.  The downstream BSP computes the
> + * first value as
> + *
> + *   clr = (~(0x00fe7f40 & status)) & (0xff0180bf & status)
> + *
> + * The two masks are complements, so that is status & 0xff0180bf: bits 0-5,
> + * 7, 15, 16 and 24-31 are kept, IRQ_RAW and the status bits 8-14 dropped.
> + *
> + * It is applied unconditionally here.  The BSP gates it on the revision in
> + * VDPU720_REG_VERSION matching 0xdb1f0006 exactly, and the RK3588 reports
> + * 0xdb1f0005, so it is not needed on that part.  The extra write is
> + * harmless there and saves a revision check that would need silicon we do
> + * not have to validate.
> + */
> +#define VDPU720_IRQ_CLR_KEEP		0xff0180bf
> +
> +#define VDPU720_REG_SYS			0x008
> +#define VDPU720_FORCE_SOFTRST		BIT(17)	/* set when dec_e=0 before soft-reset */
> +#define VDPU720_FILL_DOWN_E		BIT(24)	/* fill bottom padding rows */
> +#define VDPU720_FILL_RIGHT_E		BIT(25)
> +#define VDPU720_OUT_SEQ			BIT(26)	/* 0=raster, 1=tile */
> +#define VDPU720_YUV_OUT_FMT		GENMASK(29, 27)
> +#define VDPU720_YUV_OUT_FMT_NATIVE	0	/* no format conversion */

This is never used.

> +#define VDPU720_YUV_OUT_FMT_NV12	3	/* output as NV12 */
> +
> +#define VDPU720_REG_PIC_SIZE		0x00c
> +#define VDPU720_PIC_W_M1		GENMASK(15, 0)
> +#define VDPU720_PIC_H_M1		GENMASK(31, 16)
> +
> +#define VDPU720_REG_PIC_FMT		0x010
> +#define VDPU720_JPEG_MODE		GENMASK(2, 0)
> +#define VDPU720_JPEG_MODE_YUV400	0
> +#define VDPU720_JPEG_MODE_YUV411	1
> +#define VDPU720_JPEG_MODE_YUV420	2
> +#define VDPU720_JPEG_MODE_YUV422	3
> +#define VDPU720_JPEG_MODE_YUV440	4
> +#define VDPU720_JPEG_MODE_YUV444	5

So the driver can decode 4:4:4 JPEGs but always subsamples and outputs
them as 4:2:0 (NV12)?

> +#define VDPU720_PIX_DEPTH		GENMASK(6, 4)
> +#define VDPU720_PIX_DEPTH_8		0
> +#define VDPU720_PIX_DEPTH_12		1
> +/* qtables_sel: number of Q-table sets (0..3) */
> +#define VDPU720_QTBL_SEL		GENMASK(9, 8)
> +/* htables_sel: number of H-table sets (0..3) */
> +#define VDPU720_HTBL_SEL		GENMASK(13, 12)
> +/* dri_e: restart interval enable */
> +#define VDPU720_DRI_E			BIT(15)
> +/* dri_mcu_num_m1: restart interval MCU count minus 1 */
> +#define VDPU720_DRI_MCU_M1		GENMASK(31, 16)
> +
> +#define VDPU720_REG_HOR_STRIDE		0x014
> +#define VDPU720_Y_HOR_STRIDE		GENMASK(15, 0)
> +#define VDPU720_UV_HOR_STRIDE		GENMASK(31, 16)
> +
> +#define VDPU720_REG_Y_VSTRIDE		0x018
> +#define VDPU720_Y_VSTRIDE		GENMASK(31, 4)
> +
> +#define VDPU720_REG_TBL_LEN		0x01c
> +#define VDPU720_QTBL_LEN		GENMASK(4, 0)
> +#define VDPU720_HTBL_MINCODE_LEN	GENMASK(12, 8)
> +#define VDPU720_HTBL_VALUE_LEN		GENMASK(21, 16)
> +/* bit 16 of the Y horizontal stride, low 16 bits live in REG5 */
> +#define VDPU720_Y_HOR_STRIDE_H		BIT(24)
> +
> +#define VDPU720_REG_STRM_LEN		0x020
> +#define VDPU720_STRM_START_BYTE		GENMASK(3, 0)
> +#define VDPU720_STRM_LEN		GENMASK(31, 4)
> +
> +#define VDPU720_REG_QTBL_BASE		0x024	/* Q-table side buffer,   64-byte aligned */
> +#define VDPU720_REG_HTBL_MINCODE	0x028	/* H-mincode table,       64-byte aligned */
> +#define VDPU720_REG_HTBL_VALUE		0x02c	/* H-value table,         64-byte aligned */
> +#define VDPU720_REG_STRM_BASE		0x030	/* JPEG entropy stream,   16-byte aligned */
> +#define VDPU720_REG_OUT_BASE		0x034	/* NV12 output buffer,    64-byte aligned */
> +
> +#define VDPU720_REG_STRM_ERR		0x038
> +#define VDPU720_ERROR_PRC_MODE		BIT(0)
> +#define VDPU720_STRM_FFFF_ERR_MODE	GENMASK(6, 5)
> +#define VDPU720_STRM_OTHER_MODE		GENMASK(8, 7)
> +/* Recommended default: accept errors, skip 0xFFFF, skip unknown markers */
> +#define VDPU720_STRM_ERR_DFLT		(VDPU720_ERROR_PRC_MODE | \
> +					 FIELD_PREP_CONST(VDPU720_STRM_FFFF_ERR_MODE, 2) | \
> +					 FIELD_PREP_CONST(VDPU720_STRM_OTHER_MODE, 2))
> +
> +#define VDPU720_REG_CLK_GATE		0x040
> +#define VDPU720_CLK_GATE_ALL		0xff
> +
> +#define VDPU720_REG_PERF_CTRL		0x078
> +#define VDPU720_PERF_WORK_E		BIT(0)
> +#define VDPU720_PERF_CLR_E		BIT(1)
> +#define VDPU720_PERF_CNT_TYPE		BIT(3)
> +#define VDPU720_PERF_RD_LAT_ID		GENMASK(7, 4)
> +
> +#define VDPU720_REG_AXI_CFG		0x07c
> +#define VDPU720_ADDR_ALIGN_TYPE		GENMASK(1, 0)
> +#define VDPU720_AR_CNT_ID_TYPE		BIT(2)	/* 1 = count sw_ar_count_id only */
> +#define VDPU720_AW_CNT_ID_TYPE		BIT(3)	/* 1 = count sw_aw_count_id only */
> +#define VDPU720_AR_COUNT_ID		GENMASK(7, 4)
> +#define VDPU720_AW_COUNT_ID		GENMASK(11, 8)
> +#define VDPU720_RD_TOTAL_BYTES_MODE	BIT(12)	/* 1 = count sw_ar_count_id bytes only */
> +
> +#define VDPU720_REG_DBG_MCU_POS		0x080
> +#define VDPU720_DBG_MCU_POS_X		GENMASK(15, 0)	/* column in MCU units */
> +#define VDPU720_DBG_MCU_POS_Y		GENMASK(31, 16)	/* row in MCU units */
> +
> +#define VDPU720_REG_DBG_ERROR		0x084
> +#define VDPU720_DERR_DRI_SEQ		BIT(0)	/* DRI not at expected sequence */
> +#define VDPU720_DERR_STREAM_R0		BIT(1)	/* special marker 0 detected */
> +#define VDPU720_DERR_STREAM_R1		BIT(2)	/* special marker 1 detected */
> +#define VDPU720_DERR_STREAM_FFFF	BIT(3)	/* 0xFFFF sequence in stream */
> +#define VDPU720_DERR_OTHER_MARK		BIT(4)	/* unknown JPEG marker */
> +#define VDPU720_DERR_MCU_CNT_L		BIT(8)	/* restart mark arrived too early */
> +#define VDPU720_DERR_MCU_CNT_M		BIT(9)	/* restart mark arrived too late */
> +#define VDPU720_DERR_EOI_NO_END		BIT(10)	/* EOI before frame complete */
> +#define VDPU720_DERR_END_NO_EOI		BIT(11)	/* frame complete without EOI */
> +#define VDPU720_DERR_OVERFLOW		BIT(12)	/* Huffman coefficient overflow */
> +#define VDPU720_DERR_HUFF_EMPTY		BIT(13)	/* bitstream empty before EOI */
> +#define VDPU720_DERR_FLAGS		GENMASK(13, 0)	/* all of the above */
> +#define VDPU720_DERR_FIRST_IDX		GENMASK(19, 16)	/* index of first error */
> +
> +#define VDPU720_REG_PERF_RD_MAX_LAT	0x088	/* peak read latency (clock cycles) */
> +#define VDPU720_REG_PERF_RD_LAT_SAMP	0x08c	/* read transactions above lat threshold */
> +#define VDPU720_REG_PERF_RD_LAT_ACC	0x090	/* accumulated read latency sum */
> +#define VDPU720_REG_PERF_RD_BYTES	0x094	/* total AXI read bytes this frame */
> +#define VDPU720_REG_PERF_WR_BYTES	0x098	/* total AXI write bytes this frame */
> +
> +#define VDPU720_REG_PERF_CYCLES		0x09c
> +
> +/*
> + * Side-buffer layout for Q-tables and Huffman tables
> + *
> + * The VPU720 JPEG decoder reads quantisation tables and Huffman
> + * tables from a contiguous DMA buffer with the following layout:
> + *
> + *   [0,         QTBL_SIZE):    Q-table data (u16, raster-scan order)
> + *   [HMINCODE_OFF, +HMIN_SZ):  Huffman mincode table
> + *   [HVALUE_OFF,  +HVAL_SZ):   Huffman value table
> + *
> + * The Q-tables are per component, one entry each
> + */
> +#define VDPU720_NB_COMPONENTS		3
> +#define VDPU720_QTBL_ENTRIES		64	/* 64 coefficients per table */
> +#define VDPU720_QTBL_COMP_SIZE		(VDPU720_QTBL_ENTRIES * sizeof(u16))
> +#define VDPU720_QTBL_SIZE		(VDPU720_QTBL_COMP_SIZE * VDPU720_NB_COMPONENTS)
> +
> +/*
> + * The Huffman tables are not per component and there is no per component
> + * selector register, so the mapping is fixed: the first set is used for the
> + * luma component and the second one for both chroma components.  A grayscale
> + * frame only needs the first.
> + *
> + * HTBL_SEL has a third setting, and the downstream HAL sizes the table
> + * buffer for three sets, but it only ever fills the third with a copy of the
> + * second.  Whether that set is used for the second chroma component is
> + * unknown, so it is not used here.
> + */
> +#define VDPU720_NB_HTBL_SETS		2
> +
> +/*
> + * Per-set mincode layout: 16 DC mincodes + 8 DC accaddr pairs +
> + * 16 AC mincodes + 8 AC accaddr pairs = 48 u16 = 96 bytes
> + */
> +#define VDPU720_HMINCODE_SET_SIZE	(48 * sizeof(u16))
> +#define VDPU720_HMINCODE_SIZE		(VDPU720_HMINCODE_SET_SIZE * VDPU720_NB_HTBL_SETS)
> +#define VDPU720_HMINCODE_OFF		VDPU720_QTBL_SIZE
> +
> +/* Per-set value layout: 16 DC values + 176 AC values = 192 bytes */
> +#define VDPU720_HVALUE_SET_SIZE		192
> +#define VDPU720_HVALUE_SIZE		(VDPU720_HVALUE_SET_SIZE * VDPU720_NB_HTBL_SETS)
> +#define VDPU720_HVALUE_OFF		(VDPU720_HMINCODE_OFF + \
> +					 ALIGN(VDPU720_HMINCODE_SIZE, 64))
> +
> +#define VDPU720_TABLE_BUF_SIZE		(VDPU720_HVALUE_OFF + VDPU720_HVALUE_SIZE)
> +
> +/*
> + * The three table length registers count 16 byte units, minus one.  Derive
> + * them from the sizes above so that what is programmed always matches what
> + * the driver writes into the side buffer.
> + */
> +#define VDPU720_TBL_LEN_UNIT		16
> +#define VDPU720_TBL_LEN(bytes)		((bytes) / VDPU720_TBL_LEN_UNIT - 1)
> +
> +/*
> + * Huffman value sub-layout per set (192 bytes total):
> + *   bytes [0..15]:   DC code values (up to 12 valid entries)
> + *   bytes [16..191]: AC code values (up to 162 valid entries)
> + */
> +#define VDPU720_DC_VALUES_MAX		16
> +#define VDPU720_AC_VALUES_MAX		176	/* 12*16 - 16 */
> +
> +#endif /* RKJPEGD_VDPU720_REGS_H_ */
> diff --git a/drivers/media/platform/rockchip/rkjpegd/rkjpegd.c b/drivers/media/platform/rockchip/rkjpegd/rkjpegd.c
> new file mode 100644
> index 0000000000000..b33141c33ec38
> --- /dev/null
> +++ b/drivers/media/platform/rockchip/rkjpegd/rkjpegd.c
> @@ -0,0 +1,2222 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Rockchip JPEG decoder driver
> + *
> + * The register programming is ported from the Rockchip MPP HAL
> + * (hal_jpegd_rkv.c / hal_jpegd_vpu7xx_com.c) and the downstream
> + * mpp_jpgdec.c kernel driver.
> + *
> + * Copyright (C) 2020 Rockchip Electronics Co., Ltd.
> + * Copyright (C) 2026 WolfVision GmbH
> + *   Author: Lucas Sinn <lucas.sinn@wolfvision.net>
> + * Copyright (C) 2026 Pengutronix, Sascha Hauer <s.hauer@pengutronix.de>
> + */
> +
> +#include <linux/align.h>
> +#include <linux/bitfield.h>
> +#include <linux/cleanup.h>
> +#include <linux/clk.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/iopoll.h>
> +#include <linux/kref.h>
> +#include <linux/log2.h>
> +#include <linux/module.h>
> +#include <linux/mutex.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/reset.h>
> +#include <linux/slab.h>
> +#include <linux/videodev2.h>
> +#include <linux/workqueue.h>
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fh.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-jpeg.h>
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-core.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "rkjpegd-vdpu720-regs.h"
> +
> +#define RKJPEGD_NAME "rockchip-jpegd"
> +
> +/*
> + * The reference manual gives 48x48 to 65536x65536, and a frame header cannot
> + * describe more than 65535.  The coded format steps in RKJPEGD_CODED_STEP.
> + * Beyond that only the decoded frame has to fit a 32 bit sizeimage, see
> + * rkjpegd_raw_fits().
> + */
> +#define RKJPEGD_MIN_WIDTH	48
> +#define RKJPEGD_MIN_HEIGHT	48
> +#define RKJPEGD_MAX_SIZE	65528

This seems inconsistent. Why not MIN_SIZE or MAX_WIDTH/HEIGHT?


[...]
> +/* The capture format and crop a frame decodes to. */
> +static void rkjpegd_fill_hdr_fmt(struct v4l2_pix_format_mplane *pix_mp,
> +				 struct v4l2_rect *crop,
> +				 const struct v4l2_jpeg_header *hdr)
> +{
> +	rkjpegd_fill_raw_fmt(pix_mp, rkjpegd_raw_pixelformat(hdr),
> +			     ALIGN(hdr->frame.width, RKJPEGD_RAW_STEP),
> +			     ALIGN(hdr->frame.height, RKJPEGD_RAW_STEP));

Aligning to 16 is safe, but for e.g. grayscale (or YUV 4:4:4) I'd
expect aligning to 8 to be enough (see vdpu720_mcu_width/height()).

> +	crop->left = 0;
> +	crop->top = 0;
> +	crop->width = hdr->frame.width;
> +	crop->height = hdr->frame.height;
> +}
> +
[...]
> +/*
> + * The capture format as userspace sees it.  Once the decoder stopped at a
> + * format change, that is the one of the frame it stopped in front of.
> + */
> +static void rkjpegd_get_cap_fmt(struct rkjpegd_ctx *ctx,
> +				struct v4l2_pix_format_mplane *pix_mp,
> +				struct v4l2_rect *crop)
> +{
> +	struct vb2_v4l2_buffer *src;
> +
> +	if (READ_ONCE(ctx->stopped_at_change)) {

Why does TRY_FMT depend on decoder state?

> +		src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
> +		rkjpegd_fill_hdr_fmt(pix_mp, crop,
> +				     &vb2_to_rkjpegd_src_buf(&src->vb2_buf)->header);
> +		return;
> +	}
> +
> +	*pix_mp = ctx->dst_fmt;
> +	*crop = ctx->crop;
> +}
[...]
> +static int rkjpegd_enum_framesizes(struct file *file, void *priv,
> +				   struct v4l2_frmsizeenum *fsize)
> +{
> +	/* The decoded format follows the bitstream, nothing to enumerate. */
> +	if (fsize->pixel_format == V4L2_PIX_FMT_NV12 ||
> +	    fsize->pixel_format == V4L2_PIX_FMT_GREY)
> +		return -ENOTTY;
> +
> +	if (fsize->pixel_format != V4L2_PIX_FMT_JPEG)
> +		return -EINVAL;
> +
> +	if (fsize->index)
> +		return -EINVAL;
> +
> +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +	fsize->stepwise.min_width = RKJPEGD_MIN_WIDTH;
> +	fsize->stepwise.max_width = RKJPEGD_MAX_SIZE;
> +	fsize->stepwise.step_width = RKJPEGD_CODED_STEP;
> +	fsize->stepwise.min_height = RKJPEGD_MIN_HEIGHT;
> +	fsize->stepwise.max_height = RKJPEGD_MAX_SIZE;
> +	fsize->stepwise.step_height = RKJPEGD_CODED_STEP;

This is the coded size. I'd expect this to be continuous. You just have
to make sure that the decoded buffer is properly aligned and set the
PADDED selection rectangle accordingly.

[...]
> +/*
> + * The block is not free-running: it only raises an interrupt for a job that
> + * rkjpegd_vdpu720_run() started, and that job holds a runtime PM reference
> + * until it is finished.  The registers are always clocked here.
> + */
> +static irqreturn_t rkjpegd_vdpu720_irq(int irq, void *dev_id)
> +{
> +	struct rkjpegd_dev *jpegd = dev_id;
> +	enum vb2_buffer_state state;
> +	u32 status;
> +
> +	status = rkjpegd_read(jpegd, VDPU720_REG_INT);
> +
> +	/* First phase of the IRQ clear, see VDPU720_IRQ_CLR_KEEP. */
> +	rkjpegd_write(jpegd, status & VDPU720_IRQ_CLR_KEEP, VDPU720_REG_INT);
> +
> +	if (!(status & VDPU720_IRQ_RAW))
> +		return IRQ_NONE;
> +
> +	rkjpegd_write(jpegd, 0, VDPU720_REG_INT);
> +
> +	state = (status & VDPU720_ERR_MASK) ?
> +		VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE;
> +
> +	/*
> +	 * A block that did not park itself wedges on the next frame, and a
> +	 * clean frame can leave it unparked too, as SOFT_RST_RDY shows.  The
> +	 * reset sleeps, so leave it to rkjpegd_vdpu720_run().  The next job
> +	 * is only started after job_spinlock has released this one, which
> +	 * orders the store.
> +	 */
> +	if (state == VB2_BUF_STATE_ERROR || !(status & VDPU720_SOFT_RST_RDY))
> +		jpegd->needs_reset = true;
> +
[...]
> +
> +	rkjpegd_irq_done(jpegd, state);

Why is this hidden in a separate function call?

> +	return IRQ_HANDLED;
> +}
> +
> +static void rkjpegd_watchdog(struct work_struct *work)
> +{
> +	struct rkjpegd_dev *jpegd = container_of(to_delayed_work(work),
> +						 struct rkjpegd_dev,
> +						 watchdog_work);
> +	/* Only armed while a job runs, so there is always a context. */
> +	struct rkjpegd_ctx *ctx = v4l2_m2m_get_curr_priv(jpegd->m2m_dev);
> +
> +	dev_err(jpegd->dev, "frame processing timed out\n");
> +
> +	rkjpegd_vdpu720_reset(ctx);
> +	rkjpegd_job_finish(ctx, VB2_BUF_STATE_ERROR);
> +}
[...]
> +static int rkjpegd_buf_out_validate(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +
> +	vbuf->field = V4L2_FIELD_NONE;

Hmm, "drivers can use this to validate userspace-provided information".
Is this really validation?

It seems a lot of drivers have exactly the same function:

  vpu_vb2_buf_out_validate()
  cedrus_buf_out_validate()
  mdp_m2m_buf_out_validate()
  vb2ops_venc_buf_out_validate()
  tegra_buf_out_validate()
  dw100_buf_out_validate()
  mxc_jpeg_buf_out_validate()
  rkvdec_buf_out_validate()
  hantro_buf_out_validate()
  vicodec_buf_out_validate()
  visl_buf_out_validate()

At least e5010_buf_out_validate() and
dma2d_buf_out_validate() warn or fail, respectively, if the field value
is invalid.

Is overwriting field to NONE what this callback was designed for, and
if so, does this warrant a common helper?

> +
> +	return 0;
> +}
> 
[...]
> +
> +static int rkjpegd_resume(struct device *dev)
> +{
> +	struct rkjpegd_dev *jpegd = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = pm_runtime_force_resume(dev);

I'd expect this to not call v4l2_m2m_resume() if runtime PM resume
failed.

> +	v4l2_m2m_resume(jpegd->m2m_dev);
> +
> +	return ret;
> +}

regards
Philipp

  parent reply	other threads:[~2026-11-20 14:58 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-22 11:06 [PATCH v7 0/4] " Sascha Hauer
2026-10-22 11:06 ` [PATCH v7 1/4] media: dt-bindings: Add Rockchip JPEG decoder Sascha Hauer
2026-10-22 11:06 ` [PATCH v7 2/4] media: rockchip: Add JPEG decoder driver Sascha Hauer
2026-10-22 17:00   ` Torsten Duwe
2026-10-23  6:59     ` Sascha Hauer
2026-10-28 19:53     ` Nicolas Dufresne
2026-11-07 13:38   ` Sergei Khomenkov
2026-11-18 14:27     ` Sascha Hauer
2026-11-19 16:00       ` Nicolas Dufresne
2026-11-20 13:49         ` Sascha Hauer
2026-11-19 20:19       ` Sergei Khomenkov
2026-11-20 13:54         ` Sascha Hauer
2026-11-20 14:58   ` Philipp Zabel [this message]
2026-10-22 11:06 ` [PATCH v7 3/4] arm64: dts: rockchip: rk3588: Add JPEG decoder node Sascha Hauer
2026-10-22 11:06 ` [PATCH v7 4/4] arm64: dts: rockchip: rk356x: " Sascha Hauer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=266c9fe4a9aca943c2c2be39a7dc0f3bea2df362.camel@pengutronix.de \
    --to=p.zabel@pengutronix.de \
    --cc=conor+dt@kernel.org \
    --cc=devicetree@vger.kernel.org \
    --cc=heiko@sntech.de \
    --cc=krzk+dt@kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-rockchip@lists.infradead.org \
    --cc=lucas.sinn@wolfvision.net \
    --cc=mchehab@kernel.org \
    --cc=robh@kernel.org \
    --cc=s.hauer@pengutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®