* [PATCH RFC 01/12] dt-bindings: media: Add the Rockchip VEPU580 video encoder
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 02/12] media: rockchip: Add a VEPU580 stateless video encoder driver Arun Iyengar via B4 Relay
` (10 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The VEPU580 (RKVENC) is the H.264/HEVC video encoder of the RK3588. The SoC
has two identical cores at 0xfdbd0000 and 0xfdbe0000, each with its own
clocks, resets, power domain, interrupt and IOMMU.
Describe each core as an independent node. Unlike the vendor kernel, which
groups the cores under software-only "mpp-service" and "ccu" nodes, nothing
in the hardware ties the cores together, so how they are used (one job per
core, or both cores on one stream) is left to the operating system.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
.../bindings/media/rockchip,rk3588-vepu580.yaml | 93 ++++++++++++++++++++++
MAINTAINERS | 7 ++
2 files changed, 100 insertions(+)
diff --git a/Documentation/devicetree/bindings/media/rockchip,rk3588-vepu580.yaml b/Documentation/devicetree/bindings/media/rockchip,rk3588-vepu580.yaml
new file mode 100644
index 000000000000..2b52b17de93d
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/rockchip,rk3588-vepu580.yaml
@@ -0,0 +1,93 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/rockchip,rk3588-vepu580.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Rockchip VEPU580 video encoder
+
+maintainers:
+ - Heiko Stuebner <heiko@sntech.de>
+
+description:
+ The VEPU580 (also called RKVENC) is the H.264 and HEVC video encoder of the
+ Rockchip RK3588. The SoC has two identical encoder cores, each with its own
+ clocks, resets, power domain, interrupt and IOMMU. Every core is described
+ by its own node; the operating system may use the cores independently or
+ together.
+
+properties:
+ compatible:
+ const: rockchip,rk3588-vepu580
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: AXI bus clock
+ - description: AHB bus clock
+ - description: Encoder core clock
+
+ clock-names:
+ items:
+ - const: aclk
+ - const: hclk
+ - const: core
+
+ resets:
+ items:
+ - description: AXI bus reset
+ - description: AHB bus reset
+ - description: Encoder core reset
+
+ reset-names:
+ items:
+ - const: aclk
+ - const: hclk
+ - const: core
+
+ power-domains:
+ maxItems: 1
+
+ iommus:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - power-domains
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/rockchip,rk3588-cru.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/power/rk3588-power.h>
+ #include <dt-bindings/reset/rockchip,rk3588-cru.h>
+
+ bus {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ video-codec@fdbd0000 {
+ compatible = "rockchip,rk3588-vepu580";
+ reg = <0x0 0xfdbd0000 0x0 0x6000>;
+ interrupts = <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_RKVENC0>, <&cru HCLK_RKVENC0>,
+ <&cru CLK_RKVENC0_CORE>;
+ clock-names = "aclk", "hclk", "core";
+ resets = <&cru SRST_A_RKVENC0>, <&cru SRST_H_RKVENC0>,
+ <&cru SRST_RKVENC0_CORE>;
+ reset-names = "aclk", "hclk", "core";
+ power-domains = <&power RK3588_PD_VENC0>;
+ iommus = <&vepu580_0_mmu>;
+ };
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index 93abbb315065..57d899b1229f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -24322,6 +24322,13 @@ S: Maintained
F: Documentation/devicetree/bindings/sound/rockchip,rk3576-sai.yaml
F: sound/soc/rockchip/rockchip_sai.*
+ROCKCHIP VEPU580 VIDEO ENCODER DRIVER
+M: Arun Iyengar <aiyengar1990@gmail.com>
+L: linux-media@vger.kernel.org
+L: linux-rockchip@lists.infradead.org
+S: Maintained
+F: Documentation/devicetree/bindings/media/rockchip,rk3588-vepu580.yaml
+
ROCKCHIP XPCS DRIVER
M: Coia Prant <coiaprant@gmail.com>
L: netdev@vger.kernel.org
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 02/12] media: rockchip: Add a VEPU580 stateless video encoder driver
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 01/12] dt-bindings: media: Add the Rockchip VEPU580 video encoder Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 03/12] media: rockchip: rkvenc: Prepare the VEPU580 helpers for a second codec Arun Iyengar via B4 Relay
` (9 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
Add a V4L2 m2m driver for the VEPU580 (RKVENC) encoder of the RK3588,
using the H.264 stateless encode uAPI and helpers (v4l2-h264-enc) proposed
by Paul Kocialkowski. H.264 is supported:
- Baseline, Main and High profile, CAVLC and CABAC, 8x8 transform, a
single slice per picture, I and P pictures with one reference picture
(the hardware has a single reference read port);
- input in NV12, NV12M, NV21, YUV420, YUV420M, YUYV or UYVY, up to
4096x4096 (wider pictures need the external line buffer, not handled);
- the parameter sets come from the v4l2-h264-enc RBSP writer and are put
at the start of the CAPTURE buffer; the hardware then writes the start
code, NAL header and slice header from its syntax registers, followed by
the slice data;
- reconstructed and reference pictures are driver-internal FBC buffers
with their quarter-resolution thumbnails, managed by the v4l2-h264-enc
reference code;
- direct QP and constant quality use a fixed frame QP. In CBR mode the
frame size target of the common rate control is also given to the
hardware macroblock row rate control with adaptive quantisation, which
may move the QP of each row by up to row_rc_qp_range (module
parameter, default 2, 0 for frame-level rate control only). The average
QP measured by the hardware is exposed through
V4L2_CID_MPEG_VIDEO_AVERAGE_QP. It is not fed back into the frame-level
statistics: with adaptive quantisation it sits below the frame QP and
the estimate would spiral down.
Each of the two cores is a component of a virtual master device that owns
the video device. The cores share the first core's IOMMU domain, and
v4l2_m2m_set_max_parallel_jobs() lets jobs of different contexts run on
both cores at the same time. The dual-core handshake (one stream on two
cores) is not used.
The codec specific part sits behind a small ops structure so that HEVC can
be added once a stateless HEVC encode uAPI exists.
Provenance: there is no public documentation of this block. The register
layout and meanings, and the default tuning values (RDO lambda and weight
tables, intra/ATR thresholds, motion search parameters, flat quantisation
tables, row rate control thresholds) were derived from the Rockchip MPP
user space library (Apache-2.0: vepu580_common.h, hal_h264e_vepu580.c,
hal_h264e_vepu580_reg.h, hal_h264e_vepu580_tune.c, vepu5xx_common.c).
They are written as named fields and formulas, not register dumps. The
power, watchdog, per-frame force clear and reset/IOMMU recovery sequences
follow the Rockchip BSP mpp_rkvenc2 driver (GPL-2.0) as found in an
out-of-tree port, with the interrupt status bits corrected to the layout
used by MPP (safe clear done is bit 2, watchdog bit 8).
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
MAINTAINERS | 1 +
drivers/media/platform/rockchip/Kconfig | 1 +
drivers/media/platform/rockchip/Makefile | 1 +
drivers/media/platform/rockchip/rkvenc/Kconfig | 17 +
drivers/media/platform/rockchip/rkvenc/Makefile | 7 +
.../media/platform/rockchip/rkvenc/rkvenc-h264.c | 520 +++++++
.../media/platform/rockchip/rkvenc/rkvenc-regs.h | 379 +++++
.../platform/rockchip/rkvenc/rkvenc-vepu580.c | 585 ++++++++
drivers/media/platform/rockchip/rkvenc/rkvenc.c | 1502 ++++++++++++++++++++
drivers/media/platform/rockchip/rkvenc/rkvenc.h | 236 +++
10 files changed, 3249 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 57d899b1229f..8d7de41e67a1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -24328,6 +24328,7 @@ L: linux-media@vger.kernel.org
L: linux-rockchip@lists.infradead.org
S: Maintained
F: Documentation/devicetree/bindings/media/rockchip,rk3588-vepu580.yaml
+F: drivers/media/platform/rockchip/rkvenc/
ROCKCHIP XPCS DRIVER
M: Coia Prant <coiaprant@gmail.com>
diff --git a/drivers/media/platform/rockchip/Kconfig b/drivers/media/platform/rockchip/Kconfig
index ba401d32f01b..e6afa60923f1 100644
--- a/drivers/media/platform/rockchip/Kconfig
+++ b/drivers/media/platform/rockchip/Kconfig
@@ -6,3 +6,4 @@ source "drivers/media/platform/rockchip/rga/Kconfig"
source "drivers/media/platform/rockchip/rkcif/Kconfig"
source "drivers/media/platform/rockchip/rkisp1/Kconfig"
source "drivers/media/platform/rockchip/rkvdec/Kconfig"
+source "drivers/media/platform/rockchip/rkvenc/Kconfig"
diff --git a/drivers/media/platform/rockchip/Makefile b/drivers/media/platform/rockchip/Makefile
index 0e0b2cbbd4bd..6a1d847a10a4 100644
--- a/drivers/media/platform/rockchip/Makefile
+++ b/drivers/media/platform/rockchip/Makefile
@@ -3,3 +3,4 @@ obj-y += rga/
obj-y += rkcif/
obj-y += rkisp1/
obj-y += rkvdec/
+obj-y += rkvenc/
diff --git a/drivers/media/platform/rockchip/rkvenc/Kconfig b/drivers/media/platform/rockchip/rkvenc/Kconfig
new file mode 100644
index 000000000000..62bb1c7d5cc0
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkvenc/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0
+config VIDEO_ROCKCHIP_VENC
+ tristate "Rockchip VEPU580 stateless video encoder driver"
+ depends on ARCH_ROCKCHIP || COMPILE_TEST
+ depends on VIDEO_DEV
+ select MEDIA_CONTROLLER
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
+ select V4L2_H264
+ select V4L2_H264_ENC
+ help
+ Support for the VEPU580 (RKVENC) video encoder of the Rockchip
+ RK3588, using the V4L2 stateless encoder interface. H.264 is
+ supported.
+
+ To compile this driver as a module, choose M here: the module
+ will be called rockchip-venc.
diff --git a/drivers/media/platform/rockchip/rkvenc/Makefile b/drivers/media/platform/rockchip/rkvenc/Makefile
new file mode 100644
index 000000000000..f41e2d2afdeb
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkvenc/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_VIDEO_ROCKCHIP_VENC) += rockchip-venc.o
+
+rockchip-venc-y += \
+ rkvenc.o \
+ rkvenc-h264.o \
+ rkvenc-vepu580.o
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
new file mode 100644
index 000000000000..c669a6322f29
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
@@ -0,0 +1,520 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip VEPU580 (RKVENC) H.264 encoding backend
+ *
+ * Parameter sets are generated by the V4L2 H.264 stateless encode core and
+ * written by the CPU at the start of the CAPTURE buffer. The hardware
+ * writes the start code, NAL header and slice header of each slice from
+ * the syntax registers, followed by the slice data.
+ *
+ * The hardware reads exactly one reference picture per frame, so the
+ * stream is restricted to I and P frames with a single reference frame.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/moduleparam.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+#include <media/v4l2-h264-enc.h>
+
+#include "rkvenc.h"
+
+#define RKVENC_H264_ROW_QP_RANGE 2
+
+static unsigned int row_rc_qp_range = RKVENC_H264_ROW_QP_RANGE;
+module_param(row_rc_qp_range, uint, 0644);
+MODULE_PARM_DESC(row_rc_qp_range,
+ "CBR: maximum per-row QP change by the hardware rate control (0 = frame-level rate control only)");
+
+#define RKVENC_H264_PROFILE_BASELINE 66
+#define RKVENC_H264_PROFILE_MAIN 77
+#define RKVENC_H264_PROFILE_HIGH 100
+
+struct rkvenc_h264_rec_buf {
+ struct rkvenc_rec_buf rec;
+};
+
+static int rkvenc_h264_state_constrain(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_state *state)
+{
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+
+ /* SPS: 8-bit 4:2:0 progressive, Baseline, Main or High. */
+ switch (sps->profile_idc) {
+ case RKVENC_H264_PROFILE_BASELINE:
+ case RKVENC_H264_PROFILE_MAIN:
+ case RKVENC_H264_PROFILE_HIGH:
+ break;
+ default:
+ sps->profile_idc = RKVENC_H264_PROFILE_HIGH;
+ break;
+ }
+
+ if (!sps->level_idc)
+ sps->level_idc = 51;
+
+ /*
+ * The hardware writes pic_order_cnt_lsb in its slice headers, so only
+ * picture order count type 0 can be described.
+ */
+ if (sps->pic_order_cnt_type)
+ return -EINVAL;
+
+ if (V4L2_H264_SPS_HAS_CHROMA_FORMAT(sps)) {
+ sps->chroma_format_idc = 1;
+ sps->flags &= ~(V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE |
+ V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS);
+ sps->bit_depth_luma_minus8 = 0;
+ sps->bit_depth_chroma_minus8 = 0;
+ }
+
+ sps->flags |= V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY |
+ V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE;
+ sps->flags &= ~(V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD |
+ V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED);
+
+ /* One reference read port. */
+ if (sps->max_num_ref_frames > 1)
+ sps->max_num_ref_frames = 1;
+
+ if (sps->log2_max_frame_num_minus4 > 12)
+ sps->log2_max_frame_num_minus4 = 12;
+ if (sps->log2_max_pic_order_cnt_lsb_minus4 > 12)
+ sps->log2_max_pic_order_cnt_lsb_minus4 = 12;
+
+ /* PPS */
+ if (sps->profile_idc == RKVENC_H264_PROFILE_BASELINE)
+ pps->flags &= ~V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE;
+ if (sps->profile_idc != RKVENC_H264_PROFILE_HIGH) {
+ pps->flags &= ~V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE;
+ /* second_chroma_qp_index_offset is a High profile syntax element. */
+ pps->second_chroma_qp_index_offset = pps->chroma_qp_index_offset;
+ }
+
+ pps->num_ref_idx_l0_default_active_minus1 = 0;
+ pps->num_ref_idx_l1_default_active_minus1 = 0;
+
+ /* Encode parameters */
+ encode->flags &= ~(V4L2_H264_ENCODE_FLAG_FIELD_PIC |
+ V4L2_H264_ENCODE_FLAG_BOTTOM_FIELD |
+ V4L2_H264_ENCODE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE);
+ encode->num_ref_idx_l0_active_minus1 = 0;
+ encode->num_ref_idx_l1_active_minus1 = 0;
+ encode->colour_plane_id = 0;
+
+ if (!(pps->flags & V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT)) {
+ encode->disable_deblocking_filter_idc = 0;
+ encode->slice_alpha_c0_offset_div2 = 0;
+ encode->slice_beta_offset_div2 = 0;
+ }
+
+ if (!(pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE))
+ encode->cabac_init_idc = 0;
+
+ return 0;
+}
+
+static void rkvenc_h264_rec_size(struct rkvenc_ctx *ctx, size_t *hdr,
+ size_t *body, size_t *thumb)
+{
+ const struct v4l2_pix_format_mplane *pix = &ctx->coded_fmt.fmt.pix_mp;
+ /* FBC data needs 16 extra lines for the hardware I/O. */
+ unsigned int aw = ALIGN(pix->width, 64);
+ unsigned int ah = ALIGN(pix->height, 16) + 16;
+
+ *hdr = ALIGN(aw * ah / 64, SZ_8K);
+ *body = aw * ah * 3 / 2;
+ *thumb = ALIGN(DIV_ROUND_UP(aw, 64) * DIV_ROUND_UP(ah, 64) * 256, SZ_8K);
+}
+
+static int rkvenc_h264_rec_buffer_alloc(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_rec_buffer *buffer)
+{
+ struct rkvenc_ctx *ctx = enc->private_data;
+ struct rkvenc_h264_rec_buf *buf;
+ size_t hdr, body, thumb;
+ int ret;
+
+ buf = kzalloc_obj(*buf);
+ if (!buf)
+ return -ENOMEM;
+
+ rkvenc_h264_rec_size(ctx, &hdr, &body, &thumb);
+ buf->rec.fbc_hdr_size = hdr;
+
+ ret = rkvenc_alloc_aux(ctx->dev, &buf->rec.pixel, hdr + body, false);
+ if (ret)
+ goto err_free;
+
+ ret = rkvenc_alloc_aux(ctx->dev, &buf->rec.thumb, thumb, false);
+ if (ret)
+ goto err_free_pixel;
+
+ buffer->private_data = buf;
+ return 0;
+
+err_free_pixel:
+ rkvenc_free_aux(ctx->dev, &buf->rec.pixel);
+err_free:
+ kfree(buf);
+ return ret;
+}
+
+static int rkvenc_h264_rec_buffer_free(struct v4l2_h264_enc *enc,
+ struct v4l2_h264_enc_rec_buffer *buffer)
+{
+ struct rkvenc_ctx *ctx = enc->private_data;
+ struct rkvenc_h264_rec_buf *buf = buffer->private_data;
+
+ if (!buf)
+ return -EINVAL;
+
+ rkvenc_free_aux(ctx->dev, &buf->rec.thumb);
+ rkvenc_free_aux(ctx->dev, &buf->rec.pixel);
+ kfree(buf);
+ buffer->private_data = NULL;
+ return 0;
+}
+
+static const struct v4l2_h264_enc_ops rkvenc_h264_enc_ops = {
+ .state_constrain = rkvenc_h264_state_constrain,
+ .rec_buffer_alloc = rkvenc_h264_rec_buffer_alloc,
+ .rec_buffer_free = rkvenc_h264_rec_buffer_free,
+};
+
+static int rkvenc_h264_start(struct rkvenc_ctx *ctx)
+{
+ struct rkvenc_h264_ctx *h264;
+ struct v4l2_h264_enc *enc;
+ int ret;
+
+ h264 = kzalloc_obj(*h264);
+ if (!h264)
+ return -ENOMEM;
+
+ enc = &h264->enc;
+ enc->ops = &rkvenc_h264_enc_ops;
+ enc->private_data = ctx;
+ enc->format_mplane = &ctx->coded_fmt.fmt.pix_mp;
+ enc->timeperframe = &ctx->timeperframe;
+ enc->ctrl_handler = &ctx->ctrl_hdl;
+ enc->ref_slots_count_init = 1;
+ enc->flags = V4L2_H264_ENC_FLAG_INTER_PRED |
+ V4L2_H264_ENC_FLAG_HW_SLICE_HEADER |
+ V4L2_H264_ENC_FLAG_CHROMA_QP_CR_OFFSET;
+
+ ret = v4l2_h264_enc_init(enc);
+ if (ret) {
+ kfree(h264);
+ return ret;
+ }
+
+ ctx->priv = h264;
+ return 0;
+}
+
+static void rkvenc_h264_stop(struct rkvenc_ctx *ctx)
+{
+ struct rkvenc_h264_ctx *h264 = ctx->priv;
+
+ if (!h264)
+ return;
+
+ v4l2_h264_enc_exit(&h264->enc);
+ kfree(h264);
+ ctx->priv = NULL;
+}
+
+static struct rkvenc_rec_buf *
+rkvenc_h264_rec(struct v4l2_h264_enc_rec_buffer *buffer)
+{
+ struct rkvenc_h264_rec_buf *buf = buffer->private_data;
+
+ return buf ? &buf->rec : NULL;
+}
+
+static void rkvenc_h264_setup_refs(struct rkvenc_h264_ctx *h264,
+ struct rkvenc_core *core, bool intra)
+{
+ struct v4l2_h264_enc_ref *ref = &h264->enc.ref;
+ struct rkvenc_rec_buf *rec = rkvenc_h264_rec(&ref->buffer_current);
+ struct rkvenc_rec_buf *refr = NULL;
+
+ rkvenc_set(core, VEPU580_RFPW_H_ADDR, rec->pixel.dma);
+ rkvenc_set(core, VEPU580_RFPW_B_ADDR, rec->pixel.dma + rec->fbc_hdr_size);
+ rkvenc_set(core, VEPU580_DSPW_ADDR, rec->thumb.dma);
+
+ if (!intra && ref->l0_active_count)
+ refr = rkvenc_h264_rec(&ref->buffers[ref->l0[0].index]);
+
+ if (refr) {
+ rkvenc_set(core, VEPU580_RFPR_H_ADDR, refr->pixel.dma);
+ rkvenc_set(core, VEPU580_RFPR_B_ADDR,
+ refr->pixel.dma + refr->fbc_hdr_size);
+ rkvenc_set(core, VEPU580_DSPR_ADDR, refr->thumb.dma);
+ }
+}
+
+static void rkvenc_h264_setup_syntax(struct rkvenc_core *core,
+ const struct v4l2_h264_enc_state *state)
+{
+ const struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ const struct v4l2_ctrl_h264_pps *pps = &state->pps;
+ const struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ bool intra = encode->slice_type == V4L2_H264_SLICE_TYPE_I;
+ bool idr = encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC;
+ bool cabac = pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE;
+ u32 val;
+
+ rkvenc_or(core, VEPU580_ENC_PIC,
+ FIELD_PREP(VEPU580_ENC_PIC_ENC_STND, VEPU580_ENC_STND_H264) |
+ (encode->nal_ref_idc ? VEPU580_ENC_PIC_CUR_FRM_REF : 0) |
+ VEPU580_ENC_PIC_BS_SCP);
+
+ rkvenc_set(core, VEPU580_H264_SYNT_NAL,
+ FIELD_PREP(VEPU580_H264_NAL_REF_IDC, encode->nal_ref_idc) |
+ FIELD_PREP(VEPU580_H264_NAL_UNIT_TYPE,
+ idr ? V4L2_H264_NALU_TYPE_SLICE_IDR :
+ V4L2_H264_NALU_TYPE_SLICE_NON_IDR));
+
+ rkvenc_set(core, VEPU580_H264_SYNT_SPS,
+ FIELD_PREP(VEPU580_H264_SPS_MAX_FNUM,
+ sps->log2_max_frame_num_minus4) |
+ ((sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE) ?
+ VEPU580_H264_SPS_DRCT_8X8 : 0) |
+ FIELD_PREP(VEPU580_H264_SPS_MPOC_LM4,
+ sps->log2_max_pic_order_cnt_lsb_minus4));
+
+ val = (cabac ? VEPU580_H264_PPS_ETPY_MODE : 0) |
+ FIELD_PREP(VEPU580_H264_PPS_NUM_REF0_IDX,
+ pps->num_ref_idx_l0_default_active_minus1) |
+ FIELD_PREP(VEPU580_H264_PPS_NUM_REF1_IDX,
+ pps->num_ref_idx_l1_default_active_minus1) |
+ FIELD_PREP(VEPU580_H264_PPS_PIC_INIT_QP,
+ pps->pic_init_qp_minus26 + 26) |
+ FIELD_PREP(VEPU580_H264_PPS_CB_OFST,
+ pps->chroma_qp_index_offset & 0x1f) |
+ FIELD_PREP(VEPU580_H264_PPS_CR_OFST,
+ pps->second_chroma_qp_index_offset & 0x1f);
+ if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE)
+ val |= VEPU580_H264_PPS_TRNS_8X8;
+ if (pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED)
+ val |= VEPU580_H264_PPS_CSIP_FLAG;
+ if (pps->flags & V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT)
+ val |= VEPU580_H264_PPS_DBF_CP_FLG;
+ rkvenc_set(core, VEPU580_H264_SYNT_PPS, val);
+
+ rkvenc_set(core, VEPU580_H264_SYNT_SLI0,
+ FIELD_PREP(VEPU580_H264_SLI0_SLI_TYPE,
+ intra ? VEPU580_H264_SLI_TYPE_I :
+ VEPU580_H264_SLI_TYPE_P) |
+ FIELD_PREP(VEPU580_H264_SLI0_PPS_ID,
+ encode->pic_parameter_set_id) |
+ FIELD_PREP(VEPU580_H264_SLI0_CBC_INIT_IDC,
+ encode->cabac_init_idc) |
+ FIELD_PREP(VEPU580_H264_SLI0_FRM_NUM, encode->frame_num));
+
+ rkvenc_set(core, VEPU580_H264_SYNT_SLI1,
+ FIELD_PREP(VEPU580_H264_SLI1_IDR_PID,
+ idr ? encode->idr_pic_id : 0xffff) |
+ FIELD_PREP(VEPU580_H264_SLI1_POC_LSB,
+ encode->pic_order_cnt_lsb));
+
+ rkvenc_set(core, VEPU580_H264_SYNT_SLI2,
+ FIELD_PREP(VEPU580_H264_SLI2_SLI_BETA_OFST,
+ encode->slice_beta_offset_div2 & 0xf) |
+ FIELD_PREP(VEPU580_H264_SLI2_SLI_ALPH_OFST,
+ encode->slice_alpha_c0_offset_div2 & 0xf) |
+ FIELD_PREP(VEPU580_H264_SLI2_DIS_DBLK_IDC,
+ encode->disable_deblocking_filter_idc));
+
+ /* Sliding window marking only; long-term marking on IDR. */
+ val = 0;
+ if (idr && (encode->flags & V4L2_H264_ENCODE_FLAG_NO_OUTPUT_OF_PRIOR_PICS))
+ val |= VEPU580_H264_REFM0_NOPP_FLG;
+ if (idr && (encode->flags & V4L2_H264_ENCODE_FLAG_LONG_TERM_REFERENCE))
+ val |= VEPU580_H264_REFM0_LTRF_FLG;
+ rkvenc_set(core, VEPU580_H264_SYNT_REFM0, val);
+ rkvenc_set(core, VEPU580_H264_SYNT_REFM1, 0);
+ rkvenc_set(core, VEPU580_H264_SYNT_REFM2, 0);
+
+ val = VEPU580_RDO_CFG_INTER_4X4 | VEPU580_RDO_CFG_CHRM_SPCL |
+ VEPU580_RDO_CFG_CCWA_E | VEPU580_RDO_CFG_ATR_E |
+ VEPU580_RDO_CFG_ATF_INTRA_E;
+ if (sps->profile_idc == RKVENC_H264_PROFILE_BASELINE &&
+ sps->level_idc <= 30)
+ val |= VEPU580_RDO_CFG_RECT_SIZE;
+ if (sps->profile_idc < RKVENC_H264_PROFILE_MAIN && !cabac)
+ val |= VEPU580_RDO_CFG_VLC_LMT;
+ rkvenc_set(core, VEPU580_RDO_CFG, val);
+}
+
+static int rkvenc_h264_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
+ struct rkvenc_run *run)
+{
+ struct rkvenc_h264_ctx *h264 = ctx->priv;
+ struct v4l2_h264_enc *enc = &h264->enc;
+ const struct v4l2_h264_enc_state *state = &enc->state_active;
+ const struct v4l2_h264_enc_rc *rc = &enc->rc;
+ unsigned int mbs;
+ bool intra;
+ int ret;
+
+ /* The parameter sets are written through the kernel mapping. */
+ if (!vb2_plane_vaddr(&run->dst->vb2_buf, 0))
+ return -ENOMEM;
+
+ ret = v4l2_h264_enc_step(enc, run->dst);
+ if (ret)
+ return ret;
+
+ run->hdr_bytes = v4l2_h264_enc_rbsp_bytes_count(&enc->rbsp);
+ if (run->hdr_bytes + SZ_4K > rkvenc_dst_size(run))
+ return -ENOSPC;
+
+ intra = state->encode.slice_type == V4L2_H264_SLICE_TYPE_I;
+ mbs = state->width_mbs * state->height_mbs;
+
+ /* A P frame must have a reconstructed reference to read. */
+ if (!intra && (!enc->ref.l0_active_count ||
+ !enc->ref.buffers[enc->ref.l0[0].index].private_data))
+ return -EINVAL;
+
+ vepu580_setup_common(ctx, core, run);
+ rkvenc_h264_setup_syntax(core, state);
+ rkvenc_h264_setup_refs(h264, core, intra);
+ vepu580_setup_me(core, state->width_mbs, state->height_mbs,
+ state->sps.level_idc, intra);
+ vepu580_setup_h264_tuning(ctx->dev, core, intra);
+
+ /*
+ * In CBR mode the frame size target from the common rate control is
+ * handed to the hardware row rate control, which adapts the QP of
+ * each macroblock row by up to RKVENC_H264_ROW_QP_RANGE.
+ */
+ h264->hw_rc = rc->mode && rc->mode->type == V4L2_H264_ENC_RC_TYPE_CBR &&
+ READ_ONCE(row_rc_qp_range);
+ if (h264->hw_rc)
+ vepu580_setup_row_rc(core, rc->qp, state->qp_min, state->qp_max,
+ min(READ_ONCE(row_rc_qp_range), 15U),
+ (u64)rc->size * 8,
+ mbs, state->width_mbs);
+ else
+ vepu580_setup_fixed_qp(core, rc->qp);
+
+ return 0;
+}
+
+static void rkvenc_h264_done(struct rkvenc_ctx *ctx, struct rkvenc_run *run,
+ const struct rkvenc_feedback *fb,
+ enum vb2_buffer_state *state)
+{
+ struct rkvenc_h264_ctx *h264 = ctx->priv;
+ struct v4l2_h264_enc *enc;
+ unsigned int mbs, avg_qp;
+ u64 bytes;
+
+ if (!h264 || *state != VB2_BUF_STATE_DONE)
+ return;
+
+ enc = &h264->enc;
+ bytes = run->hdr_bytes + fb->bs_len;
+ if (!fb->bs_len || bytes > rkvenc_dst_size(run)) {
+ *state = VB2_BUF_STATE_ERROR;
+ return;
+ }
+
+ vb2_set_plane_payload(&run->dst->vb2_buf, 0, bytes);
+
+ mbs = enc->state_active.width_mbs * enc->state_active.height_mbs;
+ avg_qp = mbs ? DIV_ROUND_CLOSEST(fb->qp_sum, mbs) : 0;
+
+ /*
+ * Don't feed the measured average back into the rate control
+ * statistics: with adaptive quantisation it sits a few steps below the
+ * frame QP, and the frame-level estimate then spirals down.
+ */
+ if (ctx->avg_qp_ctrl)
+ v4l2_ctrl_s_ctrl(ctx->avg_qp_ctrl, min(avg_qp, 51U));
+
+ if (v4l2_h264_enc_complete(enc, run->dst))
+ *state = VB2_BUF_STATE_ERROR;
+}
+
+static int rkvenc_h264_adjust_fmt(struct rkvenc_ctx *ctx, struct v4l2_format *f)
+{
+ return 0;
+}
+
+static const struct rkvenc_codec_ops rkvenc_h264_ops = {
+ .adjust_fmt = rkvenc_h264_adjust_fmt,
+ .start = rkvenc_h264_start,
+ .stop = rkvenc_h264_stop,
+ .run = rkvenc_h264_run,
+ .done = rkvenc_h264_done,
+};
+
+static const struct v4l2_ctrl_config rkvenc_h264_ctrl_cfgs[] = {
+ { .id = V4L2_CID_STATELESS_H264_SPS, },
+ { .id = V4L2_CID_STATELESS_H264_PPS, },
+ { .id = V4L2_CID_STATELESS_H264_ENCODE_PARAMS, },
+ {
+ .id = V4L2_CID_STATELESS_H264_START_CODE,
+ .min = V4L2_STATELESS_H264_START_CODE_ANNEX_B,
+ .max = V4L2_STATELESS_H264_START_CODE_ANNEX_B,
+ .def = V4L2_STATELESS_H264_START_CODE_ANNEX_B,
+ },
+ { .id = V4L2_CID_MPEG_VIDEO_AU_DELIMITER, .max = 1, .step = 1, },
+ { .id = V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR, .max = 1, .step = 1, },
+ { .id = V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE, .max = 1, .step = 1, },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
+ .min = V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
+ .max = V4L2_MPEG_VIDEO_BITRATE_MODE_CQ,
+ .def = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
+ .menu_skip_mask = BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_VBR),
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE,
+ .min = 10000, .max = 200000000, .step = 1, .def = 10000000,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY,
+ .min = 1, .max = 100, .step = 1, .def = 50,
+ },
+ { .id = V4L2_CID_MPEG_VIDEO_H264_MIN_QP, .max = 51, .step = 1, .def = 10, },
+ { .id = V4L2_CID_MPEG_VIDEO_H264_MAX_QP, .max = 51, .step = 1, .def = 51, },
+ { .id = V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, .max = 51, .step = 1, .def = 26, },
+ { .id = V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, .max = 51, .step = 1, .def = 28, },
+ { .id = V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE, .max = 1, .step = 1, },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC,
+ .max = V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_EXTENDED,
+ .def = V4L2_MPEG_VIDEO_H264_VUI_SAR_IDC_UNSPECIFIED,
+ },
+ { .id = V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_WIDTH, .max = 0xffff, .step = 1, },
+ { .id = V4L2_CID_MPEG_VIDEO_H264_VUI_EXT_SAR_HEIGHT, .max = 0xffff, .step = 1, },
+ { .id = V4L2_CID_MPEG_VIDEO_AVERAGE_QP, .max = 51, .step = 1, },
+};
+
+static const struct rkvenc_ctrls rkvenc_h264_ctrls = {
+ .ctrls = rkvenc_h264_ctrl_cfgs,
+ .num_ctrls = ARRAY_SIZE(rkvenc_h264_ctrl_cfgs),
+};
+
+const struct rkvenc_coded_fmt_desc rkvenc_h264_fmt_desc = {
+ .fourcc = V4L2_PIX_FMT_H264_SLICE,
+ .frmsize = {
+ .min_width = 64,
+ /* The external line buffer for wider pictures is not handled. */
+ .max_width = 4096,
+ .step_width = 2,
+ .min_height = 64,
+ .max_height = 4096,
+ .step_height = 2,
+ },
+ .ctrls = &rkvenc_h264_ctrls,
+ .ops = &rkvenc_h264_ops,
+};
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
new file mode 100644
index 000000000000..6957cdf53446
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
@@ -0,0 +1,379 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Rockchip VEPU580 (RKVENC v2) register definitions
+ *
+ * The register meanings and field layouts were taken from the Rockchip MPP
+ * user space library register structures (vepu580_common.h,
+ * hal_h264e_vepu580_reg.h). No TRM chapter was available.
+ */
+
+#ifndef RKVENC_REGS_H_
+#define RKVENC_REGS_H_
+
+#include <linux/bits.h>
+
+/* Control class */
+#define VEPU580_VERSION 0x0000
+#define VEPU580_ENC_STRT 0x0010
+#define VEPU580_ENC_STRT_LKT_NUM GENMASK(7, 0)
+#define VEPU580_ENC_STRT_VEPU_CMD GENMASK(9, 8)
+#define VEPU580_ENC_CLR 0x0014
+#define VEPU580_ENC_CLR_SAFE BIT(0)
+#define VEPU580_ENC_CLR_FORCE BIT(1)
+#define VEPU580_INT_EN 0x0020
+#define VEPU580_INT_MSK 0x0024
+#define VEPU580_INT_CLR 0x0028
+#define VEPU580_INT_STA 0x002c
+#define VEPU580_INT_ENC_DONE BIT(0)
+#define VEPU580_INT_LKT_NODE_DONE BIT(1)
+#define VEPU580_INT_SCLR_DONE BIT(2)
+#define VEPU580_INT_SLC_DONE BIT(3)
+#define VEPU580_INT_BSF_OFLW BIT(4)
+#define VEPU580_INT_BRSP_OTSD BIT(5)
+#define VEPU580_INT_WBUS_ERR BIT(6)
+#define VEPU580_INT_RBUS_ERR BIT(7)
+#define VEPU580_INT_WDG BIT(8)
+#define VEPU580_INT_LKT_ERR BIT(9)
+#define VEPU580_INT_ALL GENMASK(9, 0)
+#define VEPU580_INT_ERROR GENMASK(9, 4)
+#define VEPU580_DTRNS_MAP 0x0030
+#define VEPU580_DTRNS_MAP_LPFW_BUS_ORDR BIT(0)
+#define VEPU580_DTRNS_MAP_CMVW_BUS_ORDR BIT(1)
+#define VEPU580_DTRNS_MAP_DSPW_BUS_ORDR BIT(2)
+#define VEPU580_DTRNS_MAP_RFPW_BUS_ORDR BIT(3)
+#define VEPU580_DTRNS_MAP_SRC_BUS_EDIN GENMASK(7, 4)
+#define VEPU580_DTRNS_MAP_MEIW_BUS_EDIN GENMASK(11, 8)
+#define VEPU580_DTRNS_MAP_BSW_BUS_EDIN GENMASK(14, 12)
+#define VEPU580_DTRNS_MAP_AFBC_BSIZE BIT(27)
+#define VEPU580_DTRNS_CFG 0x0034
+#define VEPU580_DTRNS_CFG_DSPR_OTSD BIT(7)
+#define VEPU580_DTRNS_CFG_AXI_BRSP_CKE GENMASK(23, 16)
+#define VEPU580_ENC_WDG 0x0038
+#define VEPU580_ENC_WDG_VS_LOAD_THD GENMASK(23, 0)
+#define VEPU580_ENC_WDG_RFP_LOAD_THD GENMASK(31, 24)
+#define VEPU580_FUNC_EN 0x0054
+#define VEPU580_FUNC_EN_CKE BIT(0)
+#define VEPU580_FUNC_EN_RESETN_HW_EN BIT(1)
+#define VEPU580_FUNC_EN_ENC_DONE_TMVP_EN BIT(2)
+#define VEPU580_FUNC_EN_SRAM_CKG_EN BIT(3)
+#define VEPU580_RDO_CKG 0x0058
+#define VEPU580_CTL_START 0x0030
+#define VEPU580_CTL_END 0x005c
+
+/* Frame class: addresses */
+#define VEPU580_ADR_SRC0 0x0280
+#define VEPU580_ADR_SRC1 0x0284
+#define VEPU580_ADR_SRC2 0x0288
+#define VEPU580_RFPW_H_ADDR 0x028c
+#define VEPU580_RFPW_B_ADDR 0x0290
+#define VEPU580_RFPR_H_ADDR 0x0294
+#define VEPU580_RFPR_B_ADDR 0x0298
+#define VEPU580_CMVW_ADDR 0x029c
+#define VEPU580_CMVR_ADDR 0x02a0
+#define VEPU580_DSPW_ADDR 0x02a4
+#define VEPU580_DSPR_ADDR 0x02a8
+#define VEPU580_MEIW_ADDR 0x02ac
+#define VEPU580_BSBT_ADDR 0x02b0
+#define VEPU580_BSBB_ADDR 0x02b4
+#define VEPU580_BSBR_ADDR 0x02b8
+#define VEPU580_ADR_BSBS 0x02bc
+#define VEPU580_LPFW_ADDR 0x02c0
+#define VEPU580_LPFR_ADDR 0x02c4
+#define VEPU580_EBUFT_ADDR 0x02d8
+#define VEPU580_EBUFB_ADDR 0x02dc
+
+/* Frame class: control */
+#define VEPU580_ENC_PIC 0x0300
+#define VEPU580_ENC_PIC_ENC_STND BIT(0)
+#define VEPU580_ENC_STND_H264 0
+#define VEPU580_ENC_STND_HEVC 1
+#define VEPU580_ENC_PIC_ROI_EN BIT(1)
+#define VEPU580_ENC_PIC_CUR_FRM_REF BIT(2)
+#define VEPU580_ENC_PIC_MEI_STOR BIT(3)
+#define VEPU580_ENC_PIC_BS_SCP BIT(4)
+#define VEPU580_ENC_PIC_PIC_QP GENMASK(13, 8)
+#define VEPU580_ENC_PIC_NUM_PIC_TOT_CUR GENMASK(18, 14)
+#define VEPU580_ENC_PIC_LOG2_CTU_NUM GENMASK(23, 19)
+#define VEPU580_ENC_PIC_SLEN_FIFO BIT(30)
+#define VEPU580_ENC_PIC_REC_FBC_DIS BIT(31)
+#define VEPU580_DUAL_CORE 0x0304
+#define VEPU580_ENC_RSL 0x0310
+#define VEPU580_ENC_RSL_PIC_WD8_M1 GENMASK(10, 0)
+#define VEPU580_ENC_RSL_PIC_HD8_M1 GENMASK(26, 16)
+#define VEPU580_SRC_FILL 0x0314
+#define VEPU580_SRC_FILL_PIC_WFILL GENMASK(5, 0)
+#define VEPU580_SRC_FILL_PIC_HFILL GENMASK(21, 16)
+#define VEPU580_SRC_FMT 0x0318
+#define VEPU580_SRC_FMT_ALPHA_SWAP BIT(0)
+#define VEPU580_SRC_FMT_RBUV_SWAP BIT(1)
+#define VEPU580_SRC_FMT_SRC_CFMT GENMASK(5, 2)
+#define VEPU580_FMT_BGRA8888 0
+#define VEPU580_FMT_BGR888 1
+#define VEPU580_FMT_BGR565 2
+#define VEPU580_FMT_ARGB1555 3
+#define VEPU580_FMT_YUV422SP 4
+#define VEPU580_FMT_YUV422P 5
+#define VEPU580_FMT_YUV420SP 6
+#define VEPU580_FMT_YUV420P 7
+#define VEPU580_FMT_YUYV422 8
+#define VEPU580_FMT_UYVY422 9
+#define VEPU580_FMT_YUV400 10
+#define VEPU580_FMT_AYUV2BPP 11
+#define VEPU580_FMT_YUV444SP 12
+#define VEPU580_FMT_YUV444P 13
+#define VEPU580_FMT_ARGB4444 14
+#define VEPU580_FMT_AYUV1BPP 15
+#define VEPU580_SRC_FMT_SRC_RANGE BIT(6)
+#define VEPU580_SRC_FMT_OUT_FMT BIT(7)
+#define VEPU580_SRC_UDFY 0x031c
+#define VEPU580_SRC_UDFU 0x0320
+#define VEPU580_SRC_UDFV 0x0324
+#define VEPU580_SRC_UDF_WGT_B GENMASK(8, 0)
+#define VEPU580_SRC_UDF_WGT_G GENMASK(17, 9)
+#define VEPU580_SRC_UDF_WGT_R GENMASK(26, 18)
+#define VEPU580_SRC_UDFO 0x0328
+#define VEPU580_SRC_UDFO_OFST_V GENMASK(7, 0)
+#define VEPU580_SRC_UDFO_OFST_U GENMASK(15, 8)
+#define VEPU580_SRC_UDFO_OFST_Y GENMASK(20, 16)
+#define VEPU580_SRC_PROC 0x032c
+#define VEPU580_SRC_PROC_SRC_MIRR BIT(26)
+#define VEPU580_SRC_PROC_SRC_ROT GENMASK(28, 27)
+#define VEPU580_SRC_PROC_TXA_EN BIT(29)
+#define VEPU580_SRC_PROC_AFBCD_EN BIT(30)
+#define VEPU580_PIC_OFST 0x0330
+#define VEPU580_PIC_OFST_X GENMASK(13, 0)
+#define VEPU580_PIC_OFST_Y GENMASK(29, 16)
+#define VEPU580_SRC_STRD0 0x0334
+#define VEPU580_SRC_STRD0_STRD GENMASK(16, 0)
+#define VEPU580_SRC_STRD1 0x0338
+#define VEPU580_SRC_STRD1_STRD GENMASK(15, 0)
+#define VEPU580_RC_CFG 0x0350
+#define VEPU580_RC_CFG_RC_EN BIT(0)
+#define VEPU580_RC_CFG_AQ_EN BIT(1)
+#define VEPU580_RC_CFG_AQ_MODE BIT(2)
+#define VEPU580_RC_CFG_RC_CTU_NUM GENMASK(31, 12)
+#define VEPU580_RC_QP 0x0354
+#define VEPU580_RC_QP_RANGE GENMASK(19, 16)
+#define VEPU580_RC_QP_MAX GENMASK(25, 20)
+#define VEPU580_RC_QP_MIN GENMASK(31, 26)
+#define VEPU580_RC_TGT 0x0358
+#define VEPU580_RC_TGT_CTU_EBIT GENMASK(19, 0)
+#define VEPU580_SLI_SPLT 0x0360
+#define VEPU580_SLI_SPLT_EN BIT(0)
+#define VEPU580_SLI_SPLT_MODE BIT(1)
+#define VEPU580_SLI_SPLT_CPST BIT(2)
+#define VEPU580_SLI_SPLT_FLSH BIT(15)
+#define VEPU580_SLI_SPLT_MAX_NUM_M1 GENMASK(30, 16)
+#define VEPU580_SLI_BYTE 0x0364
+#define VEPU580_SLI_CNUM 0x0368
+#define VEPU580_ME_RNGE 0x0370
+#define VEPU580_ME_RNGE_CME_SRCH_H GENMASK(3, 0)
+#define VEPU580_ME_RNGE_CME_SRCH_V GENMASK(7, 4)
+#define VEPU580_ME_RNGE_RME_SRCH_H GENMASK(10, 8)
+#define VEPU580_ME_RNGE_RME_SRCH_V GENMASK(13, 11)
+#define VEPU580_ME_RNGE_DLT_FRM_NUM GENMASK(31, 16)
+#define VEPU580_ME_CFG 0x0374
+#define VEPU580_ME_CFG_PMV_MDST_H GENMASK(7, 0)
+#define VEPU580_ME_CFG_PMV_MDST_V GENMASK(15, 8)
+#define VEPU580_ME_CFG_MV_LIMIT GENMASK(17, 16)
+#define VEPU580_ME_CFG_PMV_NUM GENMASK(19, 18)
+#define VEPU580_ME_CFG_COLMV_STOR BIT(20)
+#define VEPU580_ME_CFG_COLMV_LOAD BIT(21)
+#define VEPU580_ME_CFG_RME_DIS GENMASK(24, 22)
+#define VEPU580_ME_CFG_FME_DIS GENMASK(29, 27)
+#define VEPU580_ME_CFG_LVL4_OVRD_EN BIT(31)
+#define VEPU580_ME_CACH 0x0378
+#define VEPU580_ME_CACH_CME_RAMA_MAX GENMASK(10, 0)
+#define VEPU580_ME_CACH_CME_RAMA_H GENMASK(15, 11)
+#define VEPU580_ME_CACH_CACH_L2_TAG GENMASK(17, 16)
+#define VEPU580_ME_CACH_CME_LINEBUF_W GENMASK(26, 18)
+#define VEPU580_GMV 0x0380
+#define VEPU580_ROI_EN 0x0390
+#define VEPU580_RDO_CFG 0x03a0
+#define VEPU580_RDO_CFG_RECT_SIZE BIT(0)
+#define VEPU580_RDO_CFG_INTER_4X4 BIT(1)
+#define VEPU580_RDO_CFG_ARB_SEL BIT(2)
+#define VEPU580_RDO_CFG_VLC_LMT BIT(3)
+#define VEPU580_RDO_CFG_CHRM_SPCL BIT(4)
+#define VEPU580_RDO_CFG_RDO_MASK GENMASK(12, 5)
+#define VEPU580_RDO_CFG_CCWA_E BIT(13)
+#define VEPU580_RDO_CFG_ATR_E BIT(15)
+#define VEPU580_RDO_CFG_ATF_INTRA_E BIT(19)
+#define VEPU580_RDO_CFG_SCL_LST_SEL GENMASK(21, 20)
+#define VEPU580_IPRD_CSTS 0x03a4
+#define VEPU580_IPRD_CSTS_VTHD_Y GENMASK(11, 0)
+#define VEPU580_IPRD_CSTS_VTHD_C GENMASK(27, 16)
+
+/* Frame class: H.264 syntax */
+#define VEPU580_H264_SYNT_NAL 0x03b0
+#define VEPU580_H264_NAL_REF_IDC GENMASK(1, 0)
+#define VEPU580_H264_NAL_UNIT_TYPE GENMASK(6, 2)
+#define VEPU580_H264_SYNT_SPS 0x03b4
+#define VEPU580_H264_SPS_MAX_FNUM GENMASK(3, 0)
+#define VEPU580_H264_SPS_DRCT_8X8 BIT(4)
+#define VEPU580_H264_SPS_MPOC_LM4 GENMASK(8, 5)
+#define VEPU580_H264_SYNT_PPS 0x03b8
+#define VEPU580_H264_PPS_ETPY_MODE BIT(0)
+#define VEPU580_H264_PPS_TRNS_8X8 BIT(1)
+#define VEPU580_H264_PPS_CSIP_FLAG BIT(2)
+#define VEPU580_H264_PPS_NUM_REF0_IDX GENMASK(4, 3)
+#define VEPU580_H264_PPS_NUM_REF1_IDX GENMASK(6, 5)
+#define VEPU580_H264_PPS_PIC_INIT_QP GENMASK(12, 7)
+#define VEPU580_H264_PPS_CB_OFST GENMASK(17, 13)
+#define VEPU580_H264_PPS_CR_OFST GENMASK(22, 18)
+#define VEPU580_H264_PPS_WGHT_PRED BIT(23)
+#define VEPU580_H264_PPS_DBF_CP_FLG BIT(24)
+#define VEPU580_H264_SYNT_SLI0 0x03bc
+#define VEPU580_H264_SLI0_SLI_TYPE GENMASK(1, 0)
+#define VEPU580_H264_SLI_TYPE_P 0
+#define VEPU580_H264_SLI_TYPE_I 2
+#define VEPU580_H264_SLI0_PPS_ID GENMASK(9, 2)
+#define VEPU580_H264_SLI0_DRCT_SMVP BIT(10)
+#define VEPU580_H264_SLI0_NUM_REF_OVRD BIT(11)
+#define VEPU580_H264_SLI0_CBC_INIT_IDC GENMASK(13, 12)
+#define VEPU580_H264_SLI0_FRM_NUM GENMASK(31, 16)
+#define VEPU580_H264_SYNT_SLI1 0x03c0
+#define VEPU580_H264_SLI1_IDR_PID GENMASK(15, 0)
+#define VEPU580_H264_SLI1_POC_LSB GENMASK(31, 16)
+#define VEPU580_H264_SYNT_SLI2 0x03c4
+#define VEPU580_H264_SLI2_RODR_PIC_IDX GENMASK(1, 0)
+#define VEPU580_H264_SLI2_REF_LIST0_RODR BIT(2)
+#define VEPU580_H264_SLI2_SLI_BETA_OFST GENMASK(6, 3)
+#define VEPU580_H264_SLI2_SLI_ALPH_OFST GENMASK(10, 7)
+#define VEPU580_H264_SLI2_DIS_DBLK_IDC GENMASK(12, 11)
+#define VEPU580_H264_SLI2_RODR_PIC_NUM GENMASK(31, 16)
+#define VEPU580_H264_SYNT_REFM0 0x03c8
+#define VEPU580_H264_REFM0_NOPP_FLG BIT(0)
+#define VEPU580_H264_REFM0_LTRF_FLG BIT(1)
+#define VEPU580_H264_REFM0_ARPM_FLG BIT(2)
+#define VEPU580_H264_SYNT_REFM1 0x03cc
+#define VEPU580_H264_SYNT_REFM2 0x03d0
+#define VEPU580_SLI_CFG 0x03f0
+#define VEPU580_SLI_CFG_SLI_CRS_EN BIT(31)
+#define VEPU580_TILE_POS 0x03f4
+#define VEPU580_FRAME_START 0x0280
+#define VEPU580_FRAME_END 0x03f8
+
+/* RC / KLUT class */
+#define VEPU580_RC_ADJ0 0x1000
+#define VEPU580_RC_ADJ1 0x1004
+#define VEPU580_RC_ADJ_QP(n, v) (((v) & 0x1f) << (5 * (n)))
+#define VEPU580_RC_DTHD(n) (0x1008 + 4 * (n))
+#define VEPU580_ROI_QTHD(n) (0x1030 + 4 * (n))
+#define VEPU580_MADI_CFG 0x1040
+#define VEPU580_MADI_CFG_MADI_MODE BIT(0)
+#define VEPU580_MADI_CFG_MADI_THD GENMASK(23, 16)
+#define VEPU580_AQ_TTHD(n) (0x1044 + 4 * (n))
+#define VEPU580_AQ_STEP(n) (0x1054 + 4 * (n))
+#define VEPU580_MD_SAD_THD 0x1070
+#define VEPU580_MADI_THD 0x1074
+#define VEPU580_KLUT_OFST 0x1080
+#define VEPU580_KLUT_WGT(n) (0x1084 + 4 * (n))
+#define VEPU580_KLUT_WGT_NUM 24
+#define VEPU580_KLUT_START 0x1000
+#define VEPU580_KLUT_END 0x10e4
+
+/* Parameter class (section 3) */
+#define VEPU580_LVL32_INTRA_CST_THD0 0x1700
+#define VEPU580_LVL32_INTRA_CST_THD1 0x1704
+#define VEPU580_LVL16_INTRA_CST_THD0 0x1708
+#define VEPU580_LVL16_INTRA_CST_THD1 0x170c
+#define VEPU580_LVL8_INTRA_CST_THD0 0x1710
+#define VEPU580_LVL8_INTRA_CST_THD1 0x1714
+#define VEPU580_LVL16_INTRA_UL_CST_THD 0x1718
+#define VEPU580_LVL32_INTRA_CST_WGT0 0x171c
+#define VEPU580_LVL32_INTRA_CST_WGT1 0x1720
+#define VEPU580_LVL16_INTRA_CST_WGT0 0x1724
+#define VEPU580_LVL16_INTRA_CST_WGT1 0x1728
+#define VEPU580_RDO_QUANT 0x1730
+#define VEPU580_RDO_QUANT_F_BIAS_I GENMASK(9, 0)
+#define VEPU580_RDO_QUANT_F_BIAS_P GENMASK(19, 10)
+#define VEPU580_ATR_THD0 0x1740
+#define VEPU580_ATR_THD1 0x1744
+#define VEPU580_ATR_THD1_THD2 GENMASK(11, 0)
+#define VEPU580_ATR_THD1_THDQP GENMASK(21, 16)
+#define VEPU580_ATR_THD10 0x1748
+#define VEPU580_ATR_THD11 0x174c
+#define VEPU580_LVL16_ATR_WGT 0x1750
+#define VEPU580_LVL8_ATR_WGT 0x1754
+#define VEPU580_LVL4_ATR_WGT 0x1758
+#define VEPU580_CIME_SQI_CFG 0x1760
+#define VEPU580_CIME_SQI_CFG_SAD_MOD_SEL BIT(0)
+#define VEPU580_CIME_SQI_CFG_SAD_BIG_BLOCK BIT(1)
+#define VEPU580_CIME_SQI_CFG_PMV_SET_ZERO BIT(2)
+#define VEPU580_CIME_SQI_CFG_PMV_NUM GENMASK(9, 8)
+#define VEPU580_CIME_SQI_THD 0x1764
+#define VEPU580_CIME_SQI_THD_MVD_TH0 GENMASK(8, 0)
+#define VEPU580_CIME_SQI_THD_MVD_TH1 GENMASK(18, 10)
+#define VEPU580_CIME_SQI_THD_MVD_TH2 GENMASK(28, 20)
+#define VEPU580_CIME_SQI_MULTI0 0x1768
+#define VEPU580_CIME_SQI_MULTI1 0x176c
+#define VEPU580_CIME_SQI_MULTI_LO GENMASK(9, 0)
+#define VEPU580_CIME_SQI_MULTI_HI GENMASK(25, 16)
+#define VEPU580_RIME_SQI_THD 0x1770
+#define VEPU580_RIME_SQI_THD_CIME_SAD_TH0 GENMASK(11, 0)
+#define VEPU580_RIME_SQI_THD_MVD_TH0 GENMASK(19, 16)
+#define VEPU580_RIME_SQI_THD_MVD_TH1 GENMASK(27, 24)
+#define VEPU580_RIME_SQI_MULTI 0x1774
+#define VEPU580_RIME_SQI_MULTI0 GENMASK(9, 0)
+#define VEPU580_RIME_SQI_MULTI1 GENMASK(19, 10)
+#define VEPU580_RIME_SQI_MULTI2 GENMASK(29, 20)
+#define VEPU580_FME_SQI_THD0 0x1778
+#define VEPU580_FME_SQI_THD0_PU16_TH GENMASK(11, 0)
+#define VEPU580_FME_SQI_THD0_PU32_TH GENMASK(27, 16)
+#define VEPU580_FME_SQI_THD1 0x177c
+#define VEPU580_FME_SQI_THD1_PU64_TH GENMASK(11, 0)
+#define VEPU580_FME_SQI_THD1_MOVE_LAMBDA GENMASK(19, 16)
+#define VEPU580_IPRD_WGT_QP(n) (0x1800 + 4 * (n))
+#define VEPU580_RDO_WGTA_QP(n) (0x1900 + 4 * (n))
+#define VEPU580_PARAM_START 0x1700
+#define VEPU580_PARAM_END 0x19d0
+
+/* RDO (SQI) class, H.264 subset */
+#define VEPU580_RDO_SQI_CFG 0x2000
+#define VEPU580_RDO_SQI_CFG_ATF_PSKIP_EN BIT(0)
+#define VEPU580_RDO_INTRA_CIME_THD0 0x20cc
+#define VEPU580_RDO_INTRA_CIME_THD1 0x20d0
+#define VEPU580_RDO_INTRA_VAR_THD(n) (0x20d4 + 4 * (n))
+#define VEPU580_RDO_INTRA_ATF_WGT(n) (0x20e4 + 4 * (n))
+#define VEPU580_RDO_SKIP_CIME_THD0 0x211c
+#define VEPU580_RDO_SKIP_CIME_THD1 0x2120
+#define VEPU580_RDO_SKIP_VAR_THD(n) (0x2124 + 4 * (n))
+#define VEPU580_RDO_SKIP_ATF_WGT(n) (0x2134 + 4 * (n))
+#define VEPU580_RDO_START 0x2000
+#define VEPU580_RDO_END 0x21e8
+
+/* Scaling list class */
+#define VEPU580_SCL_START 0x2200
+#define VEPU580_SCL_H264_WORDS 576
+#define VEPU580_SCL_END 0x2c9c
+
+/* Status class */
+#define VEPU580_ST_BS_LGTH_L32 0x4000
+#define VEPU580_ST_SSE_BSL 0x4004
+#define VEPU580_ST_BS_LGTH_H8 GENMASK(7, 0)
+#define VEPU580_ST_SSE_L16 GENMASK(31, 16)
+#define VEPU580_ST_SSE_H32 0x4008
+#define VEPU580_ST_QP_SUM 0x400c
+#define VEPU580_ST_ENC 0x4020
+#define VEPU580_ST_BSB 0x402c
+#define VEPU580_ST_BUS 0x4030
+#define VEPU580_ST_SNUM 0x4034
+#define VEPU580_ST_SLEN 0x4038
+#define VEPU580_ST_PNUM_I16 0x4114
+#define VEPU580_ST_PNUM_I8 0x4118
+#define VEPU580_ST_PNUM_I4 0x411c
+#define VEPU580_ST_MADP 0x4120
+#define VEPU580_ST_BNUM_CME 0x4124
+#define VEPU580_ST_MADI 0x4128
+#define VEPU580_ST_BNUM_B16 0x412c
+
+/* Debug class */
+#define VEPU580_DBG_FRAME_CYC 0x5200
+#define VEPU580_DBG_AXIP0_CMD 0x5300
+#define VEPU580_DBG_AXIP0_CLR BIT(1)
+
+/* Software shadow covers everything up to the end of the scaling lists. */
+#define VEPU580_SHADOW_SIZE VEPU580_SCL_END
+
+#endif /* RKVENC_REGS_H_ */
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
new file mode 100644
index 000000000000..33d5154ef8da
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
@@ -0,0 +1,585 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip VEPU580 (RKVENC) common frame programming
+ *
+ * The register meanings and the default tuning values below were derived
+ * from the Rockchip MPP user space library (hal_h264e_vepu580.c,
+ * vepu5xx_common.c, hal_h264e_vepu580_tune.c), which is the only
+ * documentation of this block. The values are expressed as named fields
+ * or as formulas instead of register dumps.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/dma-mapping.h>
+#include <linux/minmax.h>
+#include <linux/slab.h>
+
+#include "rkvenc.h"
+
+int rkvenc_alloc_aux(struct rkvenc_dev *rkvenc, struct rkvenc_aux_buf *buf,
+ size_t size, bool kernel_mapping)
+{
+ struct device *dev = rkvenc->main_core->dev;
+
+ buf->attrs = kernel_mapping ? 0 : DMA_ATTR_NO_KERNEL_MAPPING;
+ buf->size = size;
+ buf->cpu = dma_alloc_attrs(dev, size, &buf->dma, GFP_KERNEL, buf->attrs);
+ if (!buf->cpu)
+ return -ENOMEM;
+
+ return 0;
+}
+
+void rkvenc_free_aux(struct rkvenc_dev *rkvenc, struct rkvenc_aux_buf *buf)
+{
+ if (!buf->cpu)
+ return;
+
+ dma_free_attrs(rkvenc->main_core->dev, buf->size, buf->cpu, buf->dma,
+ buf->attrs);
+ buf->cpu = NULL;
+}
+
+dma_addr_t rkvenc_dst_addr(struct rkvenc_run *run)
+{
+ return vb2_dma_contig_plane_dma_addr(&run->dst->vb2_buf, 0);
+}
+
+size_t rkvenc_dst_size(struct rkvenc_run *run)
+{
+ return vb2_plane_size(&run->dst->vb2_buf, 0);
+}
+
+static void vepu580_setup_ctl(struct rkvenc_core *core)
+{
+ /*
+ * Bitstream writes are byte swapped within 64 bits (bsw_bus_edin = 7);
+ * every other port uses the natural order.
+ */
+ rkvenc_set(core, VEPU580_DTRNS_MAP,
+ FIELD_PREP(VEPU580_DTRNS_MAP_BSW_BUS_EDIN, 7) |
+ VEPU580_DTRNS_MAP_AFBC_BSIZE);
+ rkvenc_set(core, VEPU580_DTRNS_CFG, VEPU580_DTRNS_CFG_DSPR_OTSD);
+ rkvenc_set(core, VEPU580_FUNC_EN,
+ VEPU580_FUNC_EN_CKE | VEPU580_FUNC_EN_RESETN_HW_EN |
+ VEPU580_FUNC_EN_ENC_DONE_TMVP_EN);
+}
+
+static void vepu580_setup_src(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
+ struct rkvenc_run *run)
+{
+ const struct v4l2_pix_format_mplane *pix = &ctx->src_fmt.fmt.pix_mp;
+ const struct rkvenc_raw_fmt *fmt = ctx->raw_fmt;
+ const struct v4l2_format_info *info = v4l2_format_info(pix->pixelformat);
+ struct vb2_buffer *vb = &run->src->vb2_buf;
+ unsigned int width = ctx->coded_fmt.fmt.pix_mp.width;
+ unsigned int height = ctx->coded_fmt.fmt.pix_mp.height;
+ unsigned int ystride = pix->plane_fmt[0].bytesperline;
+ unsigned int cstride = 0;
+ dma_addr_t addr[3] = { 0 };
+ unsigned int i;
+
+ rkvenc_set(core, VEPU580_ENC_RSL,
+ FIELD_PREP(VEPU580_ENC_RSL_PIC_WD8_M1, ALIGN(width, 16) / 8 - 1) |
+ FIELD_PREP(VEPU580_ENC_RSL_PIC_HD8_M1, ALIGN(height, 16) / 8 - 1));
+ rkvenc_set(core, VEPU580_SRC_FILL,
+ FIELD_PREP(VEPU580_SRC_FILL_PIC_WFILL, ALIGN(width, 16) - width) |
+ FIELD_PREP(VEPU580_SRC_FILL_PIC_HFILL, ALIGN(height, 16) - height));
+
+ /* MPP sets src_range for every YUV input; out_fmt selects 4:2:0. */
+ rkvenc_set(core, VEPU580_SRC_FMT,
+ FIELD_PREP(VEPU580_SRC_FMT_SRC_CFMT, fmt->hw_fmt) |
+ (fmt->rbuv_swap ? VEPU580_SRC_FMT_RBUV_SWAP : 0) |
+ VEPU580_SRC_FMT_SRC_RANGE | VEPU580_SRC_FMT_OUT_FMT);
+
+ /* BT.601 limited range RGB to YUV, unused for YUV input. */
+ rkvenc_set(core, VEPU580_SRC_UDFY,
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_R, 66) |
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_G, 129) |
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_B, 25));
+ rkvenc_set(core, VEPU580_SRC_UDFU,
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_R, -38 & 0x1ff) |
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_G, -74 & 0x1ff) |
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_B, 112));
+ rkvenc_set(core, VEPU580_SRC_UDFV,
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_R, 112) |
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_G, -94 & 0x1ff) |
+ FIELD_PREP(VEPU580_SRC_UDF_WGT_B, -18 & 0x1ff));
+ rkvenc_set(core, VEPU580_SRC_UDFO,
+ FIELD_PREP(VEPU580_SRC_UDFO_OFST_Y, 16) |
+ FIELD_PREP(VEPU580_SRC_UDFO_OFST_U, 128) |
+ FIELD_PREP(VEPU580_SRC_UDFO_OFST_V, 128));
+
+ /* Plane addresses */
+ for (i = 0; i < pix->num_planes; i++)
+ addr[i] = vb2_dma_contig_plane_dma_addr(vb, i);
+
+ if (info->comp_planes > 1) {
+ if (pix->num_planes > 1)
+ cstride = pix->plane_fmt[1].bytesperline;
+ else if (info->comp_planes == 2)
+ cstride = ystride * info->bpp[1] / info->hdiv;
+ else
+ cstride = ystride / info->hdiv;
+ }
+
+ if (pix->num_planes == 1 && info->comp_planes > 1) {
+ addr[1] = addr[0] + ystride * pix->height;
+ if (info->comp_planes > 2)
+ addr[2] = addr[1] + cstride * pix->height / info->vdiv;
+ }
+
+ rkvenc_set(core, VEPU580_ADR_SRC0, addr[0]);
+ rkvenc_set(core, VEPU580_ADR_SRC1, addr[1]);
+ rkvenc_set(core, VEPU580_ADR_SRC2, addr[2]);
+ rkvenc_set(core, VEPU580_SRC_STRD0, FIELD_PREP(VEPU580_SRC_STRD0_STRD, ystride));
+ rkvenc_set(core, VEPU580_SRC_STRD1, FIELD_PREP(VEPU580_SRC_STRD1_STRD, cstride));
+ rkvenc_set(core, VEPU580_PIC_OFST, 0);
+ rkvenc_set(core, VEPU580_SRC_PROC, 0);
+}
+
+static void vepu580_setup_bitstream(struct rkvenc_core *core,
+ struct rkvenc_run *run)
+{
+ dma_addr_t base = rkvenc_dst_addr(run);
+ size_t size = rkvenc_dst_size(run);
+
+ /*
+ * The bitstream buffer is a ring: bottom/top are its bounds (top is the
+ * address of the last byte), bsbr the read pointer and bsbs the write
+ * start. Software headers occupy the first hdr_bytes bytes.
+ */
+ rkvenc_set(core, VEPU580_BSBB_ADDR, base);
+ rkvenc_set(core, VEPU580_BSBT_ADDR, base + size - 1);
+ rkvenc_set(core, VEPU580_BSBR_ADDR, base);
+ rkvenc_set(core, VEPU580_ADR_BSBS, base + run->hdr_bytes);
+}
+
+/*
+ * Clear the shadow registers and program everything that is not codec
+ * specific: control class, source picture, bitstream buffer and slices.
+ */
+void vepu580_setup_common(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
+ struct rkvenc_run *run)
+{
+ memset(core->shadow, 0, VEPU580_SHADOW_SIZE);
+
+ vepu580_setup_ctl(core);
+ vepu580_setup_src(ctx, core, run);
+ vepu580_setup_bitstream(core, run);
+
+ /* Single slice, deblocking across slice boundaries allowed. */
+ rkvenc_set(core, VEPU580_SLI_SPLT, 0);
+ rkvenc_set(core, VEPU580_SLI_CFG, VEPU580_SLI_CFG_SLI_CRS_EN);
+
+ /* No external line buffer: widths above 4096 are rejected for now. */
+ rkvenc_set(core, VEPU580_EBUFT_ADDR, 0);
+ rkvenc_set(core, VEPU580_EBUFB_ADDR, 0);
+}
+
+/*
+ * Size the coarse integer motion estimation (CIME) cache for a search
+ * window of +-4*srch_h by +-4*srch_v pixels and a zero global MV.
+ */
+static void vepu580_setup_cime_cache(struct rkvenc_core *core,
+ unsigned int width_mbs,
+ unsigned int srch_h, unsigned int srch_v)
+{
+ const unsigned int cache_max = 2464;
+ unsigned int s = 4 * srch_h, h = 4 * srch_v;
+ unsigned int cols = ALIGN(width_mbs * 16, 64) / 64;
+ unsigned int rx = (16 - s % 16) % 16;
+ unsigned int ry = (4 - h % 4) % 4;
+ unsigned int cur_w, rows0, rama_h, ramb_h;
+
+ cur_w = min(DIV_ROUND_UP(rx + 2 * s, 16) + 1, cols);
+ rows0 = DIV_ROUND_UP(ry + 2 * h, 4) + 1;
+
+ rama_h = 1;
+ ramb_h = rows0;
+ while (rama_h < rows0 &&
+ cache_max > (rama_h - 1) * cols * 4 + ramb_h * 4 * cur_w) {
+ rama_h++;
+ ramb_h = ramb_h > 2 ? ramb_h - 1 : 1;
+ }
+
+ if (rows0 == 1) {
+ rama_h++;
+ ramb_h = 0;
+ }
+
+ if (cache_max < (rama_h - 1) * cols * 4 + ramb_h * 4 * cur_w)
+ rama_h--;
+
+ rkvenc_set(core, VEPU580_ME_CACH,
+ FIELD_PREP(VEPU580_ME_CACH_CME_RAMA_MAX,
+ (rama_h - 1) * cols + cur_w) |
+ FIELD_PREP(VEPU580_ME_CACH_CME_RAMA_H, rama_h) |
+ FIELD_PREP(VEPU580_ME_CACH_CME_LINEBUF_W, cols));
+}
+
+void vepu580_setup_me(struct rkvenc_core *core, unsigned int width_mbs,
+ unsigned int height_mbs, unsigned int level_idc,
+ bool intra)
+{
+ unsigned int blk_w = 44, blk_h = 28, hal;
+ unsigned int mv_limit;
+
+ /* Level 1.0 / 1b restrict the vertical range (H.264 table A-1). */
+ if (level_idc == 9 || level_idc == 10)
+ blk_h = 12;
+
+ if (blk_w / 4 * 2 > (width_mbs * 2 + 1) / 2)
+ blk_w = (width_mbs * 2 + 1) / 2 / 2 * 4;
+
+ hal = ALIGN(height_mbs * 16, 64);
+ if (blk_h / 4 > hal / 128 * 4)
+ blk_h = hal / 128 * 16;
+
+ rkvenc_set(core, VEPU580_ME_RNGE,
+ FIELD_PREP(VEPU580_ME_RNGE_CME_SRCH_H, blk_w / 4) |
+ FIELD_PREP(VEPU580_ME_RNGE_CME_SRCH_V, blk_h / 4) |
+ FIELD_PREP(VEPU580_ME_RNGE_RME_SRCH_H, 7) |
+ FIELD_PREP(VEPU580_ME_RNGE_RME_SRCH_V, 5));
+
+ if (level_idc > 20)
+ mv_limit = 2;
+ else if (level_idc >= 11)
+ mv_limit = 1;
+ else
+ mv_limit = 0;
+
+ rkvenc_set(core, VEPU580_ME_CFG,
+ FIELD_PREP(VEPU580_ME_CFG_PMV_MDST_H, intra ? 0 : 5) |
+ FIELD_PREP(VEPU580_ME_CFG_PMV_MDST_V, intra ? 0 : 5) |
+ FIELD_PREP(VEPU580_ME_CFG_MV_LIMIT, mv_limit) |
+ FIELD_PREP(VEPU580_ME_CFG_PMV_NUM, 2));
+
+ vepu580_setup_cime_cache(core, width_mbs, blk_w / 4, blk_h / 4);
+}
+
+/*
+ * H.264 RDO lambda per QP, growing by 2^(1/3) per QP step. The hardware
+ * table is indexed by QP 0..51 and starts at entry 6 of this list.
+ */
+static const u32 vepu580_h264_lambda[58] = {
+ 3, 5, 6, 7, 9, 11, 14, 18, 22, 28, 36, 45, 57, 72, 91, 115, 145, 182,
+ 230, 290, 365, 460, 580, 731, 921, 1161, 1462, 1843, 2322, 2925, 3686,
+ 4644, 5851, 7372, 9289, 11703, 14745, 18578, 23407, 29491, 37156,
+ 46814, 58982, 74313, 93628, 117964, 148626, 187257, 235929, 297252,
+ 374514, 471859, 594505, 749029, 943718, 1189010, 1498059, 1887436,
+};
+
+/*
+ * Chroma distortion weights (about 16 * 2^(k/3), saturated to 18 bits),
+ * packed three per two registers. Intra frames start at entry 0, inter
+ * frames at entry 6.
+ */
+static const u32 vepu580_chroma_klut[45] = {
+ 16, 20, 25, 32, 40, 50, 64, 80, 101, 128, 161, 203, 256, 322, 406,
+ 512, 645, 812, 1024, 1290, 1625, 2048, 2580, 3250, 4096, 5160, 6501,
+ 8192, 10321, 13003, 16384, 20642, 26007, 32768, 41285, 52015, 65535,
+ 82570, 104030, 131070, 165141, 208896, 262143, 262143, 0,
+};
+
+/* Texture adaptive QP: MADI thresholds and QP steps. */
+static const u8 vepu580_aq_tthd[16] = {
+ 0, 0, 0, 0, 3, 3, 5, 5, 8, 8, 8, 15, 15, 20, 25, 35,
+};
+
+static const s8 vepu580_aq_step_i[16] = {
+ -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 8,
+};
+
+static const s8 vepu580_aq_step_p[16] = {
+ -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 7, 8,
+};
+
+static u32 pack4(u32 a, u32 b, u32 c, u32 d)
+{
+ return (a & 0xff) | (b & 0xff) << 8 | (c & 0xff) << 16 |
+ (d & 0xff) << 24;
+}
+
+static u32 pack2x12(u32 lo, u32 hi)
+{
+ return (lo & 0xfff) | (hi & 0xfff) << 16;
+}
+
+static void vepu580_setup_klut(struct rkvenc_core *core, bool intra)
+{
+ const u32 *w = &vepu580_chroma_klut[intra ? 0 : 6];
+ unsigned int i;
+
+ rkvenc_set(core, VEPU580_KLUT_OFST, intra ? 0 : 3);
+
+ for (i = 0; i < VEPU580_KLUT_WGT_NUM; i += 2) {
+ const u32 *t = &w[i / 2 * 3];
+
+ rkvenc_set(core, VEPU580_KLUT_WGT(i),
+ (t[0] & 0x3ffff) | (t[1] & 0x1ff) << 23);
+ rkvenc_set(core, VEPU580_KLUT_WGT(i + 1),
+ (t[1] >> 9) | t[2] << 14);
+ }
+}
+
+void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
+ struct rkvenc_core *core, bool intra)
+{
+ const s8 *aq_step = intra ? vepu580_aq_step_i : vepu580_aq_step_p;
+ unsigned int i;
+
+ /* Intra prediction cost / mode decision */
+ rkvenc_set(core, VEPU580_IPRD_CSTS,
+ FIELD_PREP(VEPU580_IPRD_CSTS_VTHD_Y, 9) |
+ FIELD_PREP(VEPU580_IPRD_CSTS_VTHD_C, 63));
+
+ vepu580_setup_klut(core, intra);
+
+ rkvenc_set(core, VEPU580_MADI_CFG, FIELD_PREP(VEPU580_MADI_CFG_MADI_THD, 25));
+ rkvenc_set(core, VEPU580_MD_SAD_THD, pack4(4, 9, 15, 0));
+ rkvenc_set(core, VEPU580_MADI_THD, pack4(4, 9, 15, 0));
+
+ for (i = 0; i < 4; i++) {
+ rkvenc_set(core, VEPU580_AQ_TTHD(i),
+ pack4(vepu580_aq_tthd[4 * i], vepu580_aq_tthd[4 * i + 1],
+ vepu580_aq_tthd[4 * i + 2], vepu580_aq_tthd[4 * i + 3]));
+ rkvenc_set(core, VEPU580_AQ_STEP(i),
+ pack4(aq_step[4 * i] & 0x3f, aq_step[4 * i + 1] & 0x3f,
+ aq_step[4 * i + 2] & 0x3f, aq_step[4 * i + 3] & 0x3f));
+ }
+
+ /* Intra cost thresholds (1, 4, 9, 36) and weights */
+ rkvenc_set(core, VEPU580_LVL32_INTRA_CST_THD0, pack2x12(1, 4));
+ rkvenc_set(core, VEPU580_LVL32_INTRA_CST_THD1, pack2x12(9, 36));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_CST_THD0, pack2x12(1, 4));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_CST_THD1, pack2x12(9, 36));
+ rkvenc_set(core, VEPU580_LVL8_INTRA_CST_THD0, pack2x12(1, 4));
+ rkvenc_set(core, VEPU580_LVL8_INTRA_CST_THD1, pack2x12(9, 36));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_UL_CST_THD, 0);
+ rkvenc_set(core, VEPU580_LVL32_INTRA_CST_WGT0, pack4(48, 60, 40, 48));
+ rkvenc_set(core, VEPU580_LVL32_INTRA_CST_WGT1, pack4(48, 60, 40, 48));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_CST_WGT0, pack4(48, 60, 40, 48));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_CST_WGT1, pack4(36, 42, 28, 32));
+
+ /* Quantisation dead zone: 683/1024 (I) or 583/1024, 171/1024 (P) */
+ rkvenc_set(core, VEPU580_RDO_QUANT,
+ FIELD_PREP(VEPU580_RDO_QUANT_F_BIAS_I, intra ? 683 : 583) |
+ FIELD_PREP(VEPU580_RDO_QUANT_F_BIAS_P, 171));
+
+ /* Adaptive transform (H.264 only) */
+ if (intra) {
+ rkvenc_set(core, VEPU580_ATR_THD0, pack2x12(1, 4));
+ rkvenc_set(core, VEPU580_ATR_THD1,
+ FIELD_PREP(VEPU580_ATR_THD1_THD2, 36) |
+ FIELD_PREP(VEPU580_ATR_THD1_THDQP, 45));
+ rkvenc_set(core, VEPU580_LVL16_ATR_WGT, pack4(16, 16, 16, 0));
+ rkvenc_set(core, VEPU580_LVL8_ATR_WGT, pack4(22, 21, 20, 0));
+ rkvenc_set(core, VEPU580_LVL4_ATR_WGT, pack4(20, 18, 16, 0));
+ } else {
+ rkvenc_set(core, VEPU580_ATR_THD0, pack2x12(4, 16));
+ rkvenc_set(core, VEPU580_ATR_THD1,
+ FIELD_PREP(VEPU580_ATR_THD1_THD2, 81) |
+ FIELD_PREP(VEPU580_ATR_THD1_THDQP, 45));
+ rkvenc_set(core, VEPU580_LVL16_ATR_WGT, pack4(22, 19, 16, 0));
+ rkvenc_set(core, VEPU580_LVL8_ATR_WGT, pack4(22, 19, 18, 0));
+ rkvenc_set(core, VEPU580_LVL4_ATR_WGT, pack4(22, 19, 16, 0));
+ }
+
+ /*
+ * Motion search quality / cost multipliers. Where MPP's tuning differs
+ * per scene mode, the values of its default (non-IPC) mode are used.
+ */
+ rkvenc_set(core, VEPU580_CIME_SQI_CFG,
+ VEPU580_CIME_SQI_CFG_SAD_BIG_BLOCK |
+ VEPU580_CIME_SQI_CFG_PMV_SET_ZERO |
+ FIELD_PREP(VEPU580_CIME_SQI_CFG_PMV_NUM, 3));
+ rkvenc_set(core, VEPU580_CIME_SQI_THD,
+ FIELD_PREP(VEPU580_CIME_SQI_THD_MVD_TH0, 32) |
+ FIELD_PREP(VEPU580_CIME_SQI_THD_MVD_TH1, 80) |
+ FIELD_PREP(VEPU580_CIME_SQI_THD_MVD_TH2, 128));
+ rkvenc_set(core, VEPU580_CIME_SQI_MULTI0,
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_LO, 4) |
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_HI, 7));
+ rkvenc_set(core, VEPU580_CIME_SQI_MULTI1,
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_LO, 20) |
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_HI, 20));
+ rkvenc_set(core, VEPU580_RIME_SQI_THD,
+ FIELD_PREP(VEPU580_RIME_SQI_THD_CIME_SAD_TH0, 50) |
+ FIELD_PREP(VEPU580_RIME_SQI_THD_MVD_TH0, 3) |
+ FIELD_PREP(VEPU580_RIME_SQI_THD_MVD_TH1, 8));
+ rkvenc_set(core, VEPU580_RIME_SQI_MULTI,
+ FIELD_PREP(VEPU580_RIME_SQI_MULTI0, 4) |
+ FIELD_PREP(VEPU580_RIME_SQI_MULTI1, 16) |
+ FIELD_PREP(VEPU580_RIME_SQI_MULTI2, 64));
+ rkvenc_set(core, VEPU580_FME_SQI_THD0,
+ FIELD_PREP(VEPU580_FME_SQI_THD0_PU16_TH, 2));
+ rkvenc_set(core, VEPU580_FME_SQI_THD1,
+ FIELD_PREP(VEPU580_FME_SQI_THD1_MOVE_LAMBDA, 1));
+
+ /* RDO lambda, intra prediction weights stay zero for H.264 */
+ for (i = 0; i < 52; i++)
+ rkvenc_set(core, VEPU580_RDO_WGTA_QP(i), vepu580_h264_lambda[i + 6]);
+
+ /* RDO intra / skip thresholds and weights */
+ rkvenc_set(core, VEPU580_RDO_SQI_CFG, VEPU580_RDO_SQI_CFG_ATF_PSKIP_EN);
+ rkvenc_set(core, VEPU580_RDO_INTRA_CIME_THD0, pack2x12(28, 44));
+ rkvenc_set(core, VEPU580_RDO_INTRA_CIME_THD1, pack2x12(72, 0));
+ rkvenc_set(core, VEPU580_RDO_INTRA_VAR_THD(0), pack2x12(25, 64));
+ rkvenc_set(core, VEPU580_RDO_INTRA_VAR_THD(1), pack2x12(25, 64));
+ rkvenc_set(core, VEPU580_RDO_INTRA_VAR_THD(2), pack2x12(70, 100));
+ rkvenc_set(core, VEPU580_RDO_INTRA_VAR_THD(3), pack2x12(70, 100));
+ rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(0), pack4(22, 21, 20, 0));
+ rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(1), pack4(21, 20, 19, 0));
+ rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(2), pack4(20, 19, 18, 0));
+ rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(3), pack4(16, 16, 16, 0));
+ rkvenc_set(core, VEPU580_RDO_SKIP_CIME_THD0, pack2x12(10, 8));
+ rkvenc_set(core, VEPU580_RDO_SKIP_CIME_THD1, pack2x12(15, 25));
+ rkvenc_set(core, VEPU580_RDO_SKIP_VAR_THD(0), pack2x12(25, 40));
+ rkvenc_set(core, VEPU580_RDO_SKIP_VAR_THD(1), pack2x12(25, 40));
+ rkvenc_set(core, VEPU580_RDO_SKIP_VAR_THD(2), pack2x12(70, 100));
+ rkvenc_set(core, VEPU580_RDO_SKIP_VAR_THD(3), pack2x12(70, 100));
+ rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(0), pack4(20, 16, 16, 16));
+ rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(1), pack4(16, 16, 16, 0));
+ rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(2), pack4(16, 16, 16, 0));
+ rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(3), pack4(16, 16, 16, 0));
+
+ /* Flat quantisation tables */
+ memcpy(&core->shadow[VEPU580_SCL_START / 4], rkvenc->h264_scl,
+ VEPU580_SCL_H264_WORDS * sizeof(u32));
+}
+
+/*
+ * Frame-level QP with the hardware row rate control: each macroblock row
+ * gets target_bits / num_blocks bits per block; the cumulative deviation
+ * from that target selects one of nine QP adjustments, bounded by
+ * qp_range around the frame QP and by [qp_min, qp_max].
+ */
+void vepu580_setup_row_rc(struct rkvenc_core *core, unsigned int qp,
+ unsigned int qp_min, unsigned int qp_max,
+ unsigned int qp_range, u64 target_bits,
+ unsigned int num_blocks, unsigned int blocks_per_row)
+{
+ static const s8 qp_adj[9] = { -2, -1, 0, 1, 2, 0, 0, 0, 1 };
+ u64 ebit16 = div_u64(target_bits << 4, num_blocks);
+ s32 row_bits, thd;
+ unsigned int i;
+
+ if (ebit16 >= 0x100000)
+ ebit16 = 0x50000;
+
+ row_bits = (ebit16 * blocks_per_row) >> 4;
+ thd = 5 * row_bits / 16;
+
+ rkvenc_or(core, VEPU580_ENC_PIC, FIELD_PREP(VEPU580_ENC_PIC_PIC_QP, qp));
+ rkvenc_set(core, VEPU580_RC_CFG,
+ VEPU580_RC_CFG_RC_EN | VEPU580_RC_CFG_AQ_EN |
+ FIELD_PREP(VEPU580_RC_CFG_RC_CTU_NUM, blocks_per_row));
+ rkvenc_set(core, VEPU580_RC_QP,
+ FIELD_PREP(VEPU580_RC_QP_RANGE, qp_range) |
+ FIELD_PREP(VEPU580_RC_QP_MAX, qp_max) |
+ FIELD_PREP(VEPU580_RC_QP_MIN, qp_min));
+ rkvenc_set(core, VEPU580_RC_TGT, FIELD_PREP(VEPU580_RC_TGT_CTU_EBIT, ebit16));
+
+ rkvenc_set(core, VEPU580_RC_ADJ0, 0);
+ rkvenc_set(core, VEPU580_RC_ADJ1, 0);
+ for (i = 0; i < 9; i++)
+ rkvenc_or(core, i < 5 ? VEPU580_RC_ADJ0 : VEPU580_RC_ADJ1,
+ VEPU580_RC_ADJ_QP(i < 5 ? i : i - 5, qp_adj[i]));
+
+ rkvenc_set(core, VEPU580_RC_DTHD(0), -4 * thd);
+ rkvenc_set(core, VEPU580_RC_DTHD(1), -thd);
+ rkvenc_set(core, VEPU580_RC_DTHD(2), thd);
+ rkvenc_set(core, VEPU580_RC_DTHD(3), 4 * thd);
+ for (i = 4; i < 9; i++)
+ rkvenc_set(core, VEPU580_RC_DTHD(i), 0x7fffffff);
+
+ for (i = 0; i < 4; i++)
+ rkvenc_set(core, VEPU580_ROI_QTHD(i), 0);
+ /* All eight ROI areas use the frame QP bounds, qpmap_mode = 1. */
+ rkvenc_set(core, VEPU580_ROI_QTHD(0),
+ qp_min | qp_max << 6 | qp_min << 12 | qp_max << 18 |
+ qp_min << 24);
+ rkvenc_set(core, VEPU580_ROI_QTHD(1),
+ qp_max | qp_min << 6 | qp_max << 12 | qp_min << 18 |
+ qp_max << 24);
+ rkvenc_set(core, VEPU580_ROI_QTHD(2),
+ qp_min | qp_max << 6 | qp_min << 12 | qp_max << 18 |
+ qp_min << 24);
+ rkvenc_set(core, VEPU580_ROI_QTHD(3), qp_max | 1 << 30);
+}
+
+void vepu580_setup_fixed_qp(struct rkvenc_core *core, unsigned int qp)
+{
+ rkvenc_or(core, VEPU580_ENC_PIC, FIELD_PREP(VEPU580_ENC_PIC_PIC_QP, qp));
+ rkvenc_set(core, VEPU580_RC_CFG, 0);
+ rkvenc_set(core, VEPU580_RC_QP,
+ FIELD_PREP(VEPU580_RC_QP_MAX, qp) |
+ FIELD_PREP(VEPU580_RC_QP_MIN, qp));
+ rkvenc_set(core, VEPU580_ROI_QTHD(0),
+ qp | qp << 6 | qp << 12 | qp << 18 | qp << 24);
+ rkvenc_set(core, VEPU580_ROI_QTHD(1),
+ qp | qp << 6 | qp << 12 | qp << 18 | qp << 24);
+ rkvenc_set(core, VEPU580_ROI_QTHD(2),
+ qp | qp << 6 | qp << 12 | qp << 18 | qp << 24);
+ rkvenc_set(core, VEPU580_ROI_QTHD(3), qp | 1 << 30);
+}
+
+/*
+ * H.264 8x8 forward quantisation multipliers (ITU-T H.264 8.5.12.1
+ * normAdjust8x8 counterparts) for QP % 6, per position class v0..v5.
+ */
+static const u16 h264_quant8[6][6] = {
+ { 13107, 11428, 20972, 12222, 16777, 15481 },
+ { 11916, 10826, 19174, 11058, 14980, 14290 },
+ { 10082, 8943, 15978, 9675, 12710, 11985 },
+ { 9362, 8228, 14913, 8931, 11984, 11259 },
+ { 8192, 7346, 13159, 7740, 10486, 9777 },
+ { 7282, 6428, 11570, 6830, 9118, 8640 },
+};
+
+static unsigned int h264_quant8_class(unsigned int i, unsigned int j)
+{
+ if (i % 4 == 0 && j % 4 == 0)
+ return 0;
+ if (i % 2 == 1 && j % 2 == 1)
+ return 1;
+ if (i % 4 == 2 && j % 4 == 2)
+ return 2;
+ if ((i % 4 == 0 && j % 2 == 1) || (i % 2 == 1 && j % 4 == 0))
+ return 3;
+ if ((i % 4 == 0 && j % 4 == 2) || (i % 4 == 2 && j % 4 == 0))
+ return 4;
+ return 5;
+}
+
+/*
+ * Build the scaling list block for flat matrices: three groups of six 8x8
+ * tables (QP % 6 = 0..5, raster order, two 16-bit entries per word). The
+ * first group holds the multipliers, the second their low byte and the
+ * third their high byte, which is the layout the MPP library programs.
+ */
+u32 *vepu580_gen_h264_scl(struct device *dev)
+{
+ u16 *tab;
+ unsigned int g, q, i, j;
+
+ tab = devm_kcalloc(dev, VEPU580_SCL_H264_WORDS, sizeof(u32), GFP_KERNEL);
+ if (!tab)
+ return NULL;
+
+ for (g = 0; g < 3; g++)
+ for (q = 0; q < 6; q++)
+ for (i = 0; i < 8; i++)
+ for (j = 0; j < 8; j++) {
+ u16 v = h264_quant8[q][h264_quant8_class(i, j)];
+
+ if (g == 1)
+ v &= 0xff;
+ else if (g == 2)
+ v >>= 8;
+ tab[(g * 6 + q) * 64 + i * 8 + j] = v;
+ }
+
+ return (u32 *)tab;
+}
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
new file mode 100644
index 000000000000..0e5afaefc0dd
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
@@ -0,0 +1,1502 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip VEPU580 (RKVENC) stateless video encoder driver
+ *
+ * The VEPU580 found in the RK3588 has two identical cores, each behind its
+ * own IOMMU. Every core is a component bound to one virtual master device
+ * that owns the V4L2 m2m device; jobs from different contexts run on
+ * whichever core is idle (frame-level scheduling across streams, the
+ * dual-core handshake is not used).
+ *
+ * The power, clock, reset and error recovery sequences follow the Rockchip
+ * BSP mpp_rkvenc2 driver as found in its out-of-tree port.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/component.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/iommu.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "rkvenc.h"
+
+#define RKVENC_NAME "rockchip-venc"
+#define RKVENC_SW_TIMEOUT_MS 1000
+
+static const struct rkvenc_raw_fmt rkvenc_raw_fmts[] = {
+ { V4L2_PIX_FMT_NV12, VEPU580_FMT_YUV420SP, 0 },
+ { V4L2_PIX_FMT_NV12M, VEPU580_FMT_YUV420SP, 0 },
+ { V4L2_PIX_FMT_NV21, VEPU580_FMT_YUV420SP, 1 },
+ { V4L2_PIX_FMT_YUV420, VEPU580_FMT_YUV420P, 0 },
+ { V4L2_PIX_FMT_YUV420M, VEPU580_FMT_YUV420P, 0 },
+ { V4L2_PIX_FMT_YUYV, VEPU580_FMT_YUYV422, 0 },
+ { V4L2_PIX_FMT_UYVY, VEPU580_FMT_UYVY422, 0 },
+};
+
+static const struct rkvenc_coded_fmt_desc *rkvenc_coded_fmts[] = {
+ &rkvenc_h264_fmt_desc,
+};
+
+static const struct rkvenc_coded_fmt_desc *rkvenc_find_coded_fmt(u32 fourcc)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(rkvenc_coded_fmts); i++)
+ if (rkvenc_coded_fmts[i]->fourcc == fourcc)
+ return rkvenc_coded_fmts[i];
+
+ return NULL;
+}
+
+static const struct rkvenc_raw_fmt *rkvenc_find_raw_fmt(u32 fourcc)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(rkvenc_raw_fmts); i++)
+ if (rkvenc_raw_fmts[i].fourcc == fourcc)
+ return &rkvenc_raw_fmts[i];
+
+ return NULL;
+}
+
+static void rkvenc_fill_coded_fmt(struct rkvenc_ctx *ctx,
+ struct v4l2_pix_format_mplane *pix)
+{
+ const struct rkvenc_coded_fmt_desc *desc;
+ u32 min_size;
+
+ desc = rkvenc_find_coded_fmt(pix->pixelformat);
+ if (!desc) {
+ desc = rkvenc_coded_fmts[0];
+ pix->pixelformat = desc->fourcc;
+ }
+
+ v4l2_apply_frmsize_constraints(&pix->width, &pix->height,
+ &desc->frmsize);
+
+ pix->num_planes = 1;
+ pix->field = V4L2_FIELD_NONE;
+ pix->plane_fmt[0].bytesperline = 0;
+
+ /* Enough for an intra frame at a very low QP. */
+ min_size = ALIGN(pix->width, 16) * ALIGN(pix->height, 16) * 3 / 2;
+ min_size = max_t(u32, min_size, SZ_256K);
+ if (pix->plane_fmt[0].sizeimage < min_size / 4)
+ pix->plane_fmt[0].sizeimage = min_size;
+ pix->plane_fmt[0].sizeimage = PAGE_ALIGN(pix->plane_fmt[0].sizeimage);
+}
+
+static void rkvenc_fill_raw_fmt(struct v4l2_pix_format_mplane *pix,
+ u32 width, u32 height)
+{
+ if (!rkvenc_find_raw_fmt(pix->pixelformat))
+ pix->pixelformat = rkvenc_raw_fmts[0].fourcc;
+
+ /*
+ * Lines are padded to 16 bytes and the allocation covers the
+ * macroblock aligned height, in case the encoder reads the padding
+ * rows. Planes follow the visible height.
+ */
+ v4l2_fill_pixfmt_mp(pix, pix->pixelformat, ALIGN(width, 16),
+ ALIGN(height, 16));
+ if (pix->num_planes == 1) {
+ const struct v4l2_format_info *info =
+ v4l2_format_info(pix->pixelformat);
+
+ /* Single plane formats need their planes at the visible height. */
+ if (info && info->comp_planes > 1) {
+ struct v4l2_pix_format_mplane tmp = *pix;
+
+ v4l2_fill_pixfmt_mp(&tmp, pix->pixelformat,
+ ALIGN(width, 16), height);
+ pix->plane_fmt[0].sizeimage =
+ max(pix->plane_fmt[0].sizeimage,
+ tmp.plane_fmt[0].sizeimage +
+ pix->plane_fmt[0].bytesperline * 16);
+ }
+ }
+ pix->width = width;
+ pix->height = height;
+ pix->field = V4L2_FIELD_NONE;
+}
+
+static void rkvenc_reset_fmts(struct rkvenc_ctx *ctx)
+{
+ struct v4l2_pix_format_mplane *coded = &ctx->coded_fmt.fmt.pix_mp;
+ struct v4l2_pix_format_mplane *raw = &ctx->src_fmt.fmt.pix_mp;
+
+ memset(&ctx->coded_fmt, 0, sizeof(ctx->coded_fmt));
+ ctx->coded_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ coded->pixelformat = rkvenc_coded_fmts[0]->fourcc;
+ coded->width = 1920;
+ coded->height = 1080;
+ rkvenc_fill_coded_fmt(ctx, coded);
+ ctx->coded_fmt_desc = rkvenc_coded_fmts[0];
+
+ memset(&ctx->src_fmt, 0, sizeof(ctx->src_fmt));
+ ctx->src_fmt.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ raw->pixelformat = rkvenc_raw_fmts[0].fourcc;
+ rkvenc_fill_raw_fmt(raw, coded->width, coded->height);
+ ctx->raw_fmt = &rkvenc_raw_fmts[0];
+
+ ctx->timeperframe.numerator = 1;
+ ctx->timeperframe.denominator = 30;
+}
+
+/* ---- ioctls ---- */
+
+static int rkvenc_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, RKVENC_NAME, sizeof(cap->driver));
+ strscpy(cap->card, "rockchip,vepu580", sizeof(cap->card));
+ return 0;
+}
+
+static int rkvenc_enum_framesizes(struct file *file, void *priv,
+ struct v4l2_frmsizeenum *fsize)
+{
+ const struct rkvenc_coded_fmt_desc *desc;
+
+ if (fsize->index)
+ return -EINVAL;
+
+ desc = rkvenc_find_coded_fmt(fsize->pixel_format);
+ if (!desc) {
+ if (!rkvenc_find_raw_fmt(fsize->pixel_format))
+ return -EINVAL;
+ desc = rkvenc_coded_fmts[0];
+ }
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise = desc->frmsize;
+ return 0;
+}
+
+static int rkvenc_enum_frameintervals(struct file *file, void *priv,
+ struct v4l2_frmivalenum *fival)
+{
+ if (fival->index)
+ return -EINVAL;
+
+ if (!rkvenc_find_coded_fmt(fival->pixel_format) &&
+ !rkvenc_find_raw_fmt(fival->pixel_format))
+ return -EINVAL;
+
+ fival->type = V4L2_FRMIVAL_TYPE_CONTINUOUS;
+ fival->stepwise.min.numerator = 1;
+ fival->stepwise.min.denominator = 240;
+ fival->stepwise.max.numerator = 1;
+ fival->stepwise.max.denominator = 1;
+ fival->stepwise.step.numerator = 1;
+ fival->stepwise.step.denominator = 1;
+ return 0;
+}
+
+static int rkvenc_enum_capture_fmt(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index >= ARRAY_SIZE(rkvenc_coded_fmts))
+ return -EINVAL;
+
+ f->pixelformat = rkvenc_coded_fmts[f->index]->fourcc;
+ f->flags = V4L2_FMT_FLAG_COMPRESSED;
+ return 0;
+}
+
+static int rkvenc_enum_output_fmt(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index >= ARRAY_SIZE(rkvenc_raw_fmts))
+ return -EINVAL;
+
+ f->pixelformat = rkvenc_raw_fmts[f->index].fourcc;
+ return 0;
+}
+
+static int rkvenc_g_capture_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(file);
+
+ *f = ctx->coded_fmt;
+ return 0;
+}
+
+static int rkvenc_g_output_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(file);
+
+ *f = ctx->src_fmt;
+ return 0;
+}
+
+static int rkvenc_try_capture_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(file);
+ const struct rkvenc_coded_fmt_desc *desc;
+
+ rkvenc_fill_coded_fmt(ctx, &f->fmt.pix_mp);
+
+ desc = rkvenc_find_coded_fmt(f->fmt.pix_mp.pixelformat);
+ if (desc->ops->adjust_fmt)
+ return desc->ops->adjust_fmt(ctx, f);
+
+ return 0;
+}
+
+static int rkvenc_try_output_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(file);
+ struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp;
+ const struct rkvenc_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ u32 width = pix->width, height = pix->height;
+
+ v4l2_apply_frmsize_constraints(&width, &height, &desc->frmsize);
+ rkvenc_fill_raw_fmt(pix, width, height);
+
+ return 0;
+}
+
+static int rkvenc_s_capture_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(file);
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct v4l2_pix_format_mplane *raw = &ctx->src_fmt.fmt.pix_mp;
+ int ret;
+
+ if (ctx->started || vb2_is_busy(v4l2_m2m_get_vq(m2m_ctx, f->type)))
+ return -EBUSY;
+
+ ret = rkvenc_try_capture_fmt(file, priv, f);
+ if (ret)
+ return ret;
+
+ /* Both queues must always describe the same picture size. */
+ if (vb2_is_busy(v4l2_m2m_get_src_vq(m2m_ctx)) &&
+ (f->fmt.pix_mp.width != raw->width ||
+ f->fmt.pix_mp.height != raw->height))
+ return -EBUSY;
+
+ ctx->coded_fmt = *f;
+ ctx->coded_fmt_desc = rkvenc_find_coded_fmt(f->fmt.pix_mp.pixelformat);
+
+ /* The raw format follows the coded size. */
+ if (!vb2_is_busy(v4l2_m2m_get_src_vq(m2m_ctx)))
+ rkvenc_fill_raw_fmt(raw, f->fmt.pix_mp.width,
+ f->fmt.pix_mp.height);
+
+ return 0;
+}
+
+static int rkvenc_s_output_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(file);
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct v4l2_pix_format_mplane *coded = &ctx->coded_fmt.fmt.pix_mp;
+ int ret;
+
+ if (ctx->started || vb2_is_busy(v4l2_m2m_get_vq(m2m_ctx, f->type)))
+ return -EBUSY;
+
+ ret = rkvenc_try_output_fmt(file, priv, f);
+ if (ret)
+ return ret;
+
+ if (vb2_is_busy(v4l2_m2m_get_dst_vq(m2m_ctx)) &&
+ (f->fmt.pix_mp.width != coded->width ||
+ f->fmt.pix_mp.height != coded->height))
+ return -EBUSY;
+
+ ctx->src_fmt = *f;
+ ctx->raw_fmt = rkvenc_find_raw_fmt(f->fmt.pix_mp.pixelformat);
+
+ /* The coded format takes the size and colorimetry of the input. */
+ if (!vb2_is_busy(v4l2_m2m_get_dst_vq(m2m_ctx))) {
+ coded->width = f->fmt.pix_mp.width;
+ coded->height = f->fmt.pix_mp.height;
+ rkvenc_fill_coded_fmt(ctx, coded);
+ }
+ coded->colorspace = f->fmt.pix_mp.colorspace;
+ coded->xfer_func = f->fmt.pix_mp.xfer_func;
+ coded->ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
+ coded->quantization = f->fmt.pix_mp.quantization;
+
+ return 0;
+}
+
+static int rkvenc_g_parm(struct file *file, void *priv,
+ struct v4l2_streamparm *sp)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(file);
+
+ if (!V4L2_TYPE_IS_OUTPUT(sp->type))
+ return -EINVAL;
+
+ sp->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ sp->parm.output.timeperframe = ctx->timeperframe;
+ return 0;
+}
+
+static int rkvenc_s_parm(struct file *file, void *priv,
+ struct v4l2_streamparm *sp)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(file);
+ struct v4l2_fract *tpf = &sp->parm.output.timeperframe;
+
+ if (!V4L2_TYPE_IS_OUTPUT(sp->type))
+ return -EINVAL;
+
+ if (!tpf->numerator || !tpf->denominator) {
+ tpf->numerator = 1;
+ tpf->denominator = 30;
+ }
+
+ ctx->timeperframe = *tpf;
+ sp->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops rkvenc_ioctl_ops = {
+ .vidioc_querycap = rkvenc_querycap,
+ .vidioc_enum_framesizes = rkvenc_enum_framesizes,
+ .vidioc_enum_frameintervals = rkvenc_enum_frameintervals,
+
+ .vidioc_try_fmt_vid_cap_mplane = rkvenc_try_capture_fmt,
+ .vidioc_try_fmt_vid_out_mplane = rkvenc_try_output_fmt,
+ .vidioc_s_fmt_vid_out_mplane = rkvenc_s_output_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = rkvenc_s_capture_fmt,
+ .vidioc_g_fmt_vid_out_mplane = rkvenc_g_output_fmt,
+ .vidioc_g_fmt_vid_cap_mplane = rkvenc_g_capture_fmt,
+ .vidioc_enum_fmt_vid_out = rkvenc_enum_output_fmt,
+ .vidioc_enum_fmt_vid_cap = rkvenc_enum_capture_fmt,
+
+ .vidioc_g_parm = rkvenc_g_parm,
+ .vidioc_s_parm = rkvenc_s_parm,
+
+ .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+ .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+ .vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+ .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+ .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+ .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+
+ .vidioc_streamon = v4l2_m2m_ioctl_streamon,
+ .vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
+};
+
+/* ---- vb2 ---- */
+
+static struct v4l2_format *rkvenc_queue_fmt(struct rkvenc_ctx *ctx,
+ struct vb2_queue *vq)
+{
+ return V4L2_TYPE_IS_OUTPUT(vq->type) ? &ctx->src_fmt : &ctx->coded_fmt;
+}
+
+static int rkvenc_queue_setup(struct vb2_queue *vq, unsigned int *num_buffers,
+ unsigned int *num_planes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct rkvenc_ctx *ctx = vb2_get_drv_priv(vq);
+ struct v4l2_pix_format_mplane *pix = &rkvenc_queue_fmt(ctx, vq)->fmt.pix_mp;
+ unsigned int i;
+
+ if (*num_planes) {
+ if (*num_planes != pix->num_planes)
+ return -EINVAL;
+
+ for (i = 0; i < pix->num_planes; i++)
+ if (sizes[i] < pix->plane_fmt[i].sizeimage)
+ return -EINVAL;
+
+ return 0;
+ }
+
+ *num_planes = pix->num_planes;
+ for (i = 0; i < pix->num_planes; i++)
+ sizes[i] = pix->plane_fmt[i].sizeimage;
+
+ return 0;
+}
+
+static int rkvenc_buf_prepare(struct vb2_buffer *vb)
+{
+ struct rkvenc_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ struct v4l2_pix_format_mplane *pix =
+ &rkvenc_queue_fmt(ctx, vb->vb2_queue)->fmt.pix_mp;
+ unsigned int i;
+
+ for (i = 0; i < pix->num_planes; i++) {
+ if (vb2_plane_size(vb, i) < pix->plane_fmt[i].sizeimage)
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void rkvenc_buf_queue(struct vb2_buffer *vb)
+{
+ struct rkvenc_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+ v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, to_vb2_v4l2_buffer(vb));
+}
+
+static int rkvenc_buf_out_validate(struct vb2_buffer *vb)
+{
+ to_vb2_v4l2_buffer(vb)->field = V4L2_FIELD_NONE;
+ return 0;
+}
+
+static void rkvenc_buf_request_complete(struct vb2_buffer *vb)
+{
+ struct rkvenc_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+ v4l2_ctrl_request_complete(vb->req_obj.req, &ctx->ctrl_hdl);
+}
+
+static void rkvenc_queue_cleanup(struct vb2_queue *vq, u32 state)
+{
+ struct rkvenc_ctx *ctx = vb2_get_drv_priv(vq);
+ struct vb2_v4l2_buffer *vbuf;
+
+ for (;;) {
+ if (V4L2_TYPE_IS_OUTPUT(vq->type))
+ vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+ else
+ vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+ if (!vbuf)
+ break;
+
+ v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
+ &ctx->ctrl_hdl);
+ v4l2_m2m_buf_done(vbuf, state);
+ }
+}
+
+static int rkvenc_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct rkvenc_ctx *ctx = vb2_get_drv_priv(q);
+ const struct rkvenc_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+ int ret;
+
+ if (ctx->started)
+ return 0;
+
+ if (ctx->src_fmt.fmt.pix_mp.width != ctx->coded_fmt.fmt.pix_mp.width ||
+ ctx->src_fmt.fmt.pix_mp.height != ctx->coded_fmt.fmt.pix_mp.height)
+ ret = -EINVAL;
+ else
+ ret = desc->ops->start ? desc->ops->start(ctx) : 0;
+ if (ret) {
+ rkvenc_queue_cleanup(q, VB2_BUF_STATE_QUEUED);
+ return ret;
+ }
+
+ ctx->started = true;
+ return 0;
+}
+
+static void rkvenc_stop_streaming(struct vb2_queue *q)
+{
+ struct rkvenc_ctx *ctx = vb2_get_drv_priv(q);
+ const struct rkvenc_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+
+ /* v4l2_m2m_streamoff() already waited for this context's job. */
+ if (ctx->started) {
+ if (desc->ops->stop)
+ desc->ops->stop(ctx);
+ ctx->started = false;
+ }
+
+ rkvenc_queue_cleanup(q, VB2_BUF_STATE_ERROR);
+}
+
+static const struct vb2_ops rkvenc_queue_ops = {
+ .queue_setup = rkvenc_queue_setup,
+ .buf_prepare = rkvenc_buf_prepare,
+ .buf_queue = rkvenc_buf_queue,
+ .buf_out_validate = rkvenc_buf_out_validate,
+ .buf_request_complete = rkvenc_buf_request_complete,
+ .start_streaming = rkvenc_start_streaming,
+ .stop_streaming = rkvenc_stop_streaming,
+};
+
+static int rkvenc_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq)
+{
+ struct rkvenc_ctx *ctx = priv;
+ struct rkvenc_dev *rkvenc = ctx->dev;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ src_vq->drv_priv = ctx;
+ src_vq->ops = &rkvenc_queue_ops;
+ src_vq->mem_ops = &vb2_dma_contig_memops;
+ src_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES |
+ DMA_ATTR_NO_KERNEL_MAPPING;
+ src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->lock = &rkvenc->vdev_lock;
+ src_vq->dev = rkvenc->v4l2_dev.dev;
+ src_vq->supports_requests = true;
+ src_vq->requires_requests = true;
+
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ /* The CPU writes the parameter sets: CAPTURE needs a kernel mapping. */
+ /*
+ * MMAP only: the headers are written through the coherent kernel
+ * mapping; imported DMABUFs would need CPU cache maintenance.
+ */
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ dst_vq->io_modes = VB2_MMAP;
+ dst_vq->drv_priv = ctx;
+ dst_vq->ops = &rkvenc_queue_ops;
+ dst_vq->mem_ops = &vb2_dma_contig_memops;
+ dst_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
+ dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->lock = &rkvenc->vdev_lock;
+ dst_vq->dev = rkvenc->v4l2_dev.dev;
+
+ return vb2_queue_init(dst_vq);
+}
+
+/* ---- hardware ---- */
+
+static void rkvenc_write_range(struct rkvenc_core *core, u32 start, u32 end)
+{
+ u32 reg;
+
+ for (reg = start; reg < end; reg += 4)
+ writel_relaxed(core->shadow[reg / 4], core->regs + reg);
+}
+
+static u32 rkvenc_wdg_threshold(struct rkvenc_core *core)
+{
+ /* Hardware watchdog per picture size, as in the BSP driver. */
+ static const struct {
+ u32 pixels;
+ u32 ms;
+ } timeouts[] = {
+ { 1920 * 1088, 50 },
+ { 2560 * 1440, 100 },
+ { 4096 * 2304, 200 },
+ { 8192 * 8192, 400 },
+ };
+ unsigned long rate = core->core_clk ? clk_get_rate(core->core_clk) : 0;
+ u32 rsl = core->shadow[VEPU580_ENC_RSL / 4];
+ u32 pixels = (FIELD_GET(VEPU580_ENC_RSL_PIC_WD8_M1, rsl) + 1) *
+ (FIELD_GET(VEPU580_ENC_RSL_PIC_HD8_M1, rsl) + 1) * 64;
+ u32 ms = 800;
+ unsigned int i;
+ u64 thd;
+
+ if (!rate)
+ return VEPU580_ENC_WDG_VS_LOAD_THD;
+
+ for (i = 0; i < ARRAY_SIZE(timeouts); i++) {
+ if (pixels <= timeouts[i].pixels) {
+ ms = timeouts[i].ms;
+ break;
+ }
+ }
+
+ /*
+ * The BSP counts in units of 256 core clock cycles, its out-of-tree
+ * port in units of 1024. Using the former errs on the long side.
+ */
+ thd = (u64)ms * (rate / 256000);
+ return min_t(u64, thd, VEPU580_ENC_WDG_VS_LOAD_THD);
+}
+
+static void rkvenc_hw_start(struct rkvenc_core *core)
+{
+ /*
+ * VEPU580 erratum: force-clear the core before every frame or stale
+ * state may cause IOMMU page faults.
+ */
+ writel(VEPU580_ENC_CLR_FORCE, core->regs + VEPU580_ENC_CLR);
+ udelay(5);
+ writel(0, core->regs + VEPU580_ENC_CLR);
+
+ writel_relaxed(VEPU580_DBG_AXIP0_CLR, core->regs + VEPU580_DBG_AXIP0_CMD);
+
+ writel_relaxed(VEPU580_INT_ENC_DONE | VEPU580_INT_ERROR,
+ core->regs + VEPU580_INT_EN);
+ writel_relaxed(0, core->regs + VEPU580_INT_MSK);
+ writel_relaxed(VEPU580_INT_ALL, core->regs + VEPU580_INT_CLR);
+
+ rkvenc_write_range(core, VEPU580_CTL_START, VEPU580_CTL_END);
+ writel_relaxed(FIELD_PREP(VEPU580_ENC_WDG_VS_LOAD_THD,
+ rkvenc_wdg_threshold(core)),
+ core->regs + VEPU580_ENC_WDG);
+ rkvenc_write_range(core, VEPU580_FRAME_START, VEPU580_FRAME_END);
+ rkvenc_write_range(core, VEPU580_KLUT_START, VEPU580_KLUT_END);
+ rkvenc_write_range(core, VEPU580_PARAM_START, VEPU580_PARAM_END);
+ rkvenc_write_range(core, VEPU580_RDO_START, VEPU580_RDO_END);
+ rkvenc_write_range(core, VEPU580_SCL_START, VEPU580_SCL_END);
+
+ schedule_delayed_work(&core->watchdog_work,
+ msecs_to_jiffies(RKVENC_SW_TIMEOUT_MS));
+
+ /* writel() orders all the relaxed writes above before the start. */
+ writel(FIELD_PREP(VEPU580_ENC_STRT_VEPU_CMD, 1),
+ core->regs + VEPU580_ENC_STRT);
+}
+
+static void rkvenc_iommu_restore(struct rkvenc_core *core)
+{
+ struct iommu_domain *domain = core->rkvenc->iommu_domain;
+ int ret;
+
+ if (!core->empty_domain || !domain)
+ return;
+
+ /*
+ * A CRU reset also clears the IOMMU. Attaching an empty domain and
+ * then the shared domain again reprograms the page table base and
+ * flushes the TLB of this core's IOMMU.
+ */
+ iommu_detach_device(domain, core->dev);
+ ret = iommu_attach_device(core->empty_domain, core->dev);
+ if (ret)
+ dev_warn(core->dev, "cannot attach empty domain: %d\n", ret);
+ iommu_detach_device(core->empty_domain, core->dev);
+ ret = iommu_attach_device(domain, core->dev);
+ if (ret)
+ dev_warn(core->dev, "cannot attach shared domain: %d\n", ret);
+}
+
+static void rkvenc_hw_reset(struct rkvenc_core *core)
+{
+ u32 sta;
+ int ret;
+
+ /* Report safe-clear completion in the status, without an interrupt. */
+ writel(VEPU580_INT_ALL, core->regs + VEPU580_INT_MSK);
+ writel(VEPU580_INT_ALL, core->regs + VEPU580_INT_EN);
+ writel(VEPU580_ENC_CLR_SAFE, core->regs + VEPU580_ENC_CLR);
+ ret = readl_poll_timeout_atomic(core->regs + VEPU580_INT_STA, sta,
+ sta & VEPU580_INT_SCLR_DONE, 1, 100);
+ writel(VEPU580_ENC_CLR_FORCE, core->regs + VEPU580_ENC_CLR);
+ udelay(5);
+ writel(0, core->regs + VEPU580_ENC_CLR);
+ writel(VEPU580_INT_ALL, core->regs + VEPU580_INT_CLR);
+ writel(0, core->regs + VEPU580_INT_STA);
+
+ if (ret && core->rstc) {
+ dev_warn(core->dev, "safe clear timed out, resetting the core\n");
+ reset_control_assert(core->rstc);
+ udelay(5);
+ reset_control_deassert(core->rstc);
+ rkvenc_iommu_restore(core);
+ }
+}
+
+static void rkvenc_read_feedback(struct rkvenc_core *core,
+ struct rkvenc_feedback *fb)
+{
+ void __iomem *r = core->regs;
+ u32 sse_bsl = readl_relaxed(r + VEPU580_ST_SSE_BSL);
+
+ fb->bs_len = readl_relaxed(r + VEPU580_ST_BS_LGTH_L32) |
+ (u64)FIELD_GET(VEPU580_ST_BS_LGTH_H8, sse_bsl) << 32;
+ fb->sse = (u64)readl_relaxed(r + VEPU580_ST_SSE_H32) << 16 |
+ FIELD_GET(VEPU580_ST_SSE_L16, sse_bsl);
+ fb->qp_sum = readl_relaxed(r + VEPU580_ST_QP_SUM);
+ fb->madi = readl_relaxed(r + VEPU580_ST_MADI);
+ fb->madp = readl_relaxed(r + VEPU580_ST_MADP);
+ fb->intra_blocks = readl_relaxed(r + VEPU580_ST_PNUM_I16) +
+ readl_relaxed(r + VEPU580_ST_PNUM_I8) +
+ readl_relaxed(r + VEPU580_ST_PNUM_I4);
+ fb->cycles = readl_relaxed(r + VEPU580_DBG_FRAME_CYC);
+}
+
+static void rkvenc_release_core(struct rkvenc_core *core)
+{
+ struct rkvenc_dev *rkvenc = core->rkvenc;
+
+ guard(spinlock_irqsave)(&rkvenc->cores_lock);
+ core->curr_ctx = NULL;
+ set_bit(core->id, &rkvenc->idle_cores);
+}
+
+static struct rkvenc_core *rkvenc_acquire_core(struct rkvenc_dev *rkvenc,
+ struct rkvenc_ctx *ctx)
+{
+ struct rkvenc_core *core;
+ unsigned int id;
+
+ guard(spinlock_irqsave)(&rkvenc->cores_lock);
+ id = find_first_bit(&rkvenc->idle_cores, rkvenc->core_count);
+ if (id >= rkvenc->core_count)
+ return NULL;
+
+ clear_bit(id, &rkvenc->idle_cores);
+ core = rkvenc->cores[id];
+ core->curr_ctx = ctx;
+ return core;
+}
+
+static void rkvenc_job_finish(struct rkvenc_core *core,
+ enum vb2_buffer_state state,
+ const struct rkvenc_feedback *fb, bool powered)
+{
+ struct rkvenc_ctx *ctx = core->curr_ctx;
+ struct rkvenc_run *run = &core->run;
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct vb2_v4l2_buffer *src = run->src, *dst = run->dst;
+ struct media_request *req = src->vb2_buf.req_obj.req;
+ struct rkvenc_feedback empty = { };
+
+ if (ctx->coded_fmt_desc->ops->done)
+ ctx->coded_fmt_desc->ops->done(ctx, run, fb ? fb : &empty, &state);
+
+ if (state != VB2_BUF_STATE_DONE)
+ vb2_set_plane_payload(&dst->vb2_buf, 0, 0);
+
+ if (powered) {
+ pm_runtime_mark_last_busy(core->dev);
+ pm_runtime_put_autosuspend(core->dev);
+ }
+
+ /*
+ * Everything needed from core->run is in locals now: once the core is
+ * idle another context's device_run() may take it.
+ */
+ ctx->core = NULL;
+ rkvenc_release_core(core);
+
+ v4l2_m2m_src_buf_remove_by_buf(m2m_ctx, src);
+ v4l2_m2m_dst_buf_remove_by_buf(m2m_ctx, dst);
+ if (req)
+ v4l2_ctrl_request_complete(req, &ctx->ctrl_hdl);
+ /* CAPTURE first: the request completes with the OUTPUT buffer. */
+ v4l2_m2m_buf_done(dst, state);
+ v4l2_m2m_buf_done(src, state);
+ v4l2_m2m_job_finish(ctx->dev->m2m_dev, m2m_ctx);
+}
+
+static void rkvenc_device_run(void *priv)
+{
+ struct rkvenc_ctx *ctx = priv;
+ struct rkvenc_dev *rkvenc = ctx->dev;
+ const struct rkvenc_codec_ops *ops = ctx->coded_fmt_desc->ops;
+ struct vb2_v4l2_buffer *src, *dst;
+ struct media_request *req;
+ struct rkvenc_core *core;
+ int ret;
+
+ src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+ dst = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+
+ core = rkvenc_acquire_core(rkvenc, ctx);
+ if (WARN_ON(!core)) {
+ if (src->vb2_buf.req_obj.req)
+ v4l2_ctrl_request_complete(src->vb2_buf.req_obj.req,
+ &ctx->ctrl_hdl);
+ v4l2_m2m_buf_done_and_job_finish(rkvenc->m2m_dev,
+ ctx->fh.m2m_ctx,
+ VB2_BUF_STATE_ERROR);
+ return;
+ }
+
+ ctx->core = core;
+ core->run.src = src;
+ core->run.dst = dst;
+ core->run.hdr_bytes = 0;
+
+ req = src->vb2_buf.req_obj.req;
+ if (req)
+ v4l2_ctrl_request_setup(req, &ctx->ctrl_hdl);
+
+ /* The reference handling keys reconstructed frames by timestamp. */
+ v4l2_m2m_buf_copy_metadata(src, dst);
+
+ ret = pm_runtime_resume_and_get(core->dev);
+ if (ret < 0) {
+ dev_err(core->dev, "cannot power up: %d\n", ret);
+ rkvenc_job_finish(core, VB2_BUF_STATE_ERROR, NULL, false);
+ return;
+ }
+
+ ret = ops->run(ctx, core, &core->run);
+ if (ret) {
+ dev_dbg(core->dev, "cannot prepare job: %d\n", ret);
+ rkvenc_job_finish(core, VB2_BUF_STATE_ERROR, NULL, true);
+ return;
+ }
+
+ rkvenc_hw_start(core);
+}
+
+static int rkvenc_job_ready(void *priv)
+{
+ struct rkvenc_ctx *ctx = priv;
+
+ return ctx->started;
+}
+
+static const struct v4l2_m2m_ops rkvenc_m2m_ops = {
+ .device_run = rkvenc_device_run,
+ .job_ready = rkvenc_job_ready,
+};
+
+static irqreturn_t rkvenc_irq_thread(int irq, void *priv)
+{
+ struct rkvenc_core *core = priv;
+ enum vb2_buffer_state state = VB2_BUF_STATE_DONE;
+ struct rkvenc_feedback fb = { };
+ u32 sta;
+
+ if (pm_runtime_get_if_active(core->dev) <= 0)
+ return IRQ_NONE;
+
+ sta = readl(core->regs + VEPU580_INT_STA);
+ writel(sta, core->regs + VEPU580_INT_CLR);
+ writel(0, core->regs + VEPU580_INT_STA);
+
+ if (!sta) {
+ pm_runtime_put_noidle(core->dev);
+ return IRQ_NONE;
+ }
+
+ core->irq_count++;
+
+ /* Prevent a watchdog interrupt storm. */
+ if (sta & VEPU580_INT_WDG)
+ writel(VEPU580_INT_WDG, core->regs + VEPU580_INT_MSK);
+
+ if (!(sta & (VEPU580_INT_ENC_DONE | VEPU580_INT_ERROR)) ||
+ !core->curr_ctx || !cancel_delayed_work(&core->watchdog_work)) {
+ pm_runtime_put_noidle(core->dev);
+ return IRQ_HANDLED;
+ }
+
+ fb.int_sta = sta;
+ rkvenc_read_feedback(core, &fb);
+
+ if (sta & VEPU580_INT_ERROR) {
+ dev_err(core->dev, "encode error, irq status %#x\n", sta);
+ state = VB2_BUF_STATE_ERROR;
+ rkvenc_hw_reset(core);
+ }
+
+ pm_runtime_put_noidle(core->dev);
+ rkvenc_job_finish(core, state, &fb, true);
+
+ return IRQ_HANDLED;
+}
+
+static void rkvenc_watchdog_func(struct work_struct *work)
+{
+ struct rkvenc_core *core = container_of(to_delayed_work(work),
+ struct rkvenc_core,
+ watchdog_work);
+
+ if (!core->curr_ctx)
+ return;
+
+ dev_err(core->dev, "frame encoding timed out, irq status %#x\n",
+ readl(core->regs + VEPU580_INT_STA));
+ rkvenc_hw_reset(core);
+ rkvenc_job_finish(core, VB2_BUF_STATE_ERROR, NULL, true);
+}
+
+/* ---- file operations ---- */
+
+static int rkvenc_init_ctrls(struct rkvenc_ctx *ctx)
+{
+ struct v4l2_ctrl_handler *hdl = &ctx->ctrl_hdl;
+ unsigned int i, j, nctrls = 0;
+ int ret;
+
+ for (i = 0; i < ARRAY_SIZE(rkvenc_coded_fmts); i++)
+ nctrls += rkvenc_coded_fmts[i]->ctrls->num_ctrls;
+
+ v4l2_ctrl_handler_init(hdl, nctrls);
+
+ for (i = 0; i < ARRAY_SIZE(rkvenc_coded_fmts); i++) {
+ const struct rkvenc_ctrls *ctrls = rkvenc_coded_fmts[i]->ctrls;
+
+ for (j = 0; j < ctrls->num_ctrls; j++)
+ v4l2_ctrl_new_custom(hdl, &ctrls->ctrls[j], NULL);
+ }
+
+ ctx->avg_qp_ctrl = v4l2_ctrl_find(hdl, V4L2_CID_MPEG_VIDEO_AVERAGE_QP);
+
+ if (hdl->error) {
+ ret = hdl->error;
+ v4l2_ctrl_handler_free(hdl);
+ return ret;
+ }
+
+ ret = v4l2_ctrl_handler_setup(hdl);
+ if (ret) {
+ v4l2_ctrl_handler_free(hdl);
+ return ret;
+ }
+
+ ctx->fh.ctrl_handler = hdl;
+ return 0;
+}
+
+static int rkvenc_open(struct file *filp)
+{
+ struct rkvenc_dev *rkvenc = video_drvdata(filp);
+ struct rkvenc_ctx *ctx;
+ int ret;
+
+ ctx = kzalloc_obj(*ctx);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->dev = rkvenc;
+ rkvenc_reset_fmts(ctx);
+ v4l2_fh_init(&ctx->fh, video_devdata(filp));
+
+ ret = rkvenc_init_ctrls(ctx);
+ if (ret)
+ goto err_free_ctx;
+
+ ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(rkvenc->m2m_dev, ctx,
+ rkvenc_queue_init);
+ if (IS_ERR(ctx->fh.m2m_ctx)) {
+ ret = PTR_ERR(ctx->fh.m2m_ctx);
+ goto err_free_ctrls;
+ }
+
+ v4l2_fh_add(&ctx->fh, filp);
+ return 0;
+
+err_free_ctrls:
+ v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+err_free_ctx:
+ v4l2_fh_exit(&ctx->fh);
+ kfree(ctx);
+ return ret;
+}
+
+static int rkvenc_release(struct file *filp)
+{
+ struct rkvenc_ctx *ctx = file_to_rkvenc_ctx(filp);
+
+ v4l2_fh_del(&ctx->fh, filp);
+ v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+ v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
+ v4l2_fh_exit(&ctx->fh);
+ kfree(ctx);
+ return 0;
+}
+
+static const struct v4l2_file_operations rkvenc_fops = {
+ .owner = THIS_MODULE,
+ .open = rkvenc_open,
+ .release = rkvenc_release,
+ .poll = v4l2_m2m_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = v4l2_m2m_fop_mmap,
+};
+
+static int rkvenc_request_validate(struct media_request *req)
+{
+ unsigned int count = vb2_request_buffer_cnt(req);
+
+ if (!count)
+ return -ENOENT;
+ if (count > 1)
+ return -EINVAL;
+
+ return vb2_request_validate(req);
+}
+
+static const struct media_device_ops rkvenc_media_ops = {
+ .req_validate = rkvenc_request_validate,
+ .req_queue = v4l2_m2m_request_queue,
+};
+
+static int rkvenc_v4l2_init(struct rkvenc_dev *rkvenc)
+{
+ struct device *dev = rkvenc->main_core->dev;
+ int ret;
+
+ ret = v4l2_device_register(dev, &rkvenc->v4l2_dev);
+ if (ret)
+ return ret;
+
+ rkvenc->m2m_dev = v4l2_m2m_init(&rkvenc_m2m_ops);
+ if (IS_ERR(rkvenc->m2m_dev)) {
+ ret = PTR_ERR(rkvenc->m2m_dev);
+ goto err_unregister_v4l2;
+ }
+
+ rkvenc->mdev.dev = dev;
+ strscpy(rkvenc->mdev.model, RKVENC_NAME, sizeof(rkvenc->mdev.model));
+ /* Must match the bus_info VIDIOC_QUERYCAP reports for the vdev. */
+ snprintf(rkvenc->mdev.bus_info, sizeof(rkvenc->mdev.bus_info),
+ "platform:%s", dev_name(dev));
+ media_device_init(&rkvenc->mdev);
+ rkvenc->mdev.ops = &rkvenc_media_ops;
+ rkvenc->v4l2_dev.mdev = &rkvenc->mdev;
+
+ rkvenc->vdev.lock = &rkvenc->vdev_lock;
+ rkvenc->vdev.v4l2_dev = &rkvenc->v4l2_dev;
+ rkvenc->vdev.fops = &rkvenc_fops;
+ rkvenc->vdev.release = video_device_release_empty;
+ rkvenc->vdev.vfl_dir = VFL_DIR_M2M;
+ rkvenc->vdev.device_caps = V4L2_CAP_STREAMING |
+ V4L2_CAP_VIDEO_M2M_MPLANE;
+ rkvenc->vdev.ioctl_ops = &rkvenc_ioctl_ops;
+ video_set_drvdata(&rkvenc->vdev, rkvenc);
+ strscpy(rkvenc->vdev.name, RKVENC_NAME, sizeof(rkvenc->vdev.name));
+
+ ret = video_register_device(&rkvenc->vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ goto err_cleanup_mc;
+
+ ret = v4l2_m2m_register_media_controller(rkvenc->m2m_dev, &rkvenc->vdev,
+ MEDIA_ENT_F_PROC_VIDEO_ENCODER);
+ if (ret)
+ goto err_unregister_vdev;
+
+ ret = media_device_register(&rkvenc->mdev);
+ if (ret)
+ goto err_unregister_mc;
+
+ v4l2_m2m_set_max_parallel_jobs(rkvenc->m2m_dev, rkvenc->core_count);
+ return 0;
+
+err_unregister_mc:
+ v4l2_m2m_unregister_media_controller(rkvenc->m2m_dev);
+err_unregister_vdev:
+ video_unregister_device(&rkvenc->vdev);
+err_cleanup_mc:
+ media_device_cleanup(&rkvenc->mdev);
+ v4l2_m2m_release(rkvenc->m2m_dev);
+err_unregister_v4l2:
+ v4l2_device_unregister(&rkvenc->v4l2_dev);
+ return ret;
+}
+
+static void rkvenc_v4l2_cleanup(struct rkvenc_dev *rkvenc)
+{
+ media_device_unregister(&rkvenc->mdev);
+ v4l2_m2m_unregister_media_controller(rkvenc->m2m_dev);
+ video_unregister_device(&rkvenc->vdev);
+ media_device_cleanup(&rkvenc->mdev);
+ v4l2_m2m_release(rkvenc->m2m_dev);
+ v4l2_device_unregister(&rkvenc->v4l2_dev);
+}
+
+/* ---- cores ---- */
+
+static int rkvenc_core_bind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct rkvenc_core *core = dev_get_drvdata(dev);
+ struct rkvenc_dev *rkvenc = data;
+ int ret;
+
+ if (rkvenc->core_count >= rkvenc->num_cores)
+ return -EINVAL;
+
+ /*
+ * All cores share the first core's DMA domain, so that buffers
+ * allocated or imported through the first core are mapped in every
+ * core's IOMMU and a job can run on any core.
+ */
+ if (iommu_get_domain_for_dev(dev)) {
+ if (!rkvenc->iommu_domain) {
+ rkvenc->iommu_domain = iommu_get_domain_for_dev(dev);
+ } else {
+ ret = iommu_attach_device(rkvenc->iommu_domain, dev);
+ if (ret) {
+ dev_err(dev, "cannot attach shared IOMMU domain: %d\n",
+ ret);
+ return ret;
+ }
+ }
+ }
+
+ core->id = rkvenc->core_count;
+ core->rkvenc = rkvenc;
+ rkvenc->cores[core->id] = core;
+ if (!core->id)
+ rkvenc->main_core = core;
+ rkvenc->core_count++;
+ set_bit(core->id, &rkvenc->idle_cores);
+
+ dev_info(dev, "registered encoder core %d\n", core->id);
+ return 0;
+}
+
+static void rkvenc_core_unbind(struct device *dev, struct device *master,
+ void *data)
+{
+ struct rkvenc_core *core = dev_get_drvdata(dev);
+ struct rkvenc_dev *rkvenc = data;
+
+ cancel_delayed_work_sync(&core->watchdog_work);
+
+ if (core->id && rkvenc->iommu_domain)
+ iommu_detach_device(rkvenc->iommu_domain, dev);
+
+ clear_bit(core->id, &rkvenc->idle_cores);
+ rkvenc->cores[core->id] = NULL;
+ rkvenc->core_count--;
+ core->rkvenc = NULL;
+}
+
+static const struct component_ops rkvenc_core_ops = {
+ .bind = rkvenc_core_bind,
+ .unbind = rkvenc_core_unbind,
+};
+
+static void rkvenc_free_empty_domain(void *data)
+{
+ iommu_domain_free(data);
+}
+
+static int rkvenc_core_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct rkvenc_core *core;
+ int ret, irq;
+
+ core = devm_kzalloc(dev, sizeof(*core), GFP_KERNEL);
+ if (!core)
+ return -ENOMEM;
+
+ core->dev = dev;
+ INIT_DELAYED_WORK(&core->watchdog_work, rkvenc_watchdog_func);
+
+ core->shadow = devm_kzalloc(dev, VEPU580_SHADOW_SIZE, GFP_KERNEL);
+ if (!core->shadow)
+ return -ENOMEM;
+
+ core->regs = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(core->regs))
+ return PTR_ERR(core->regs);
+
+ ret = devm_clk_bulk_get_all(dev, &core->clocks);
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "cannot get clocks\n");
+ core->num_clocks = ret;
+
+ core->core_clk = devm_clk_get(dev, "core");
+ if (IS_ERR(core->core_clk))
+ return dev_err_probe(dev, PTR_ERR(core->core_clk),
+ "cannot get core clock\n");
+
+ core->rstc = devm_reset_control_array_get_optional_exclusive(dev);
+ if (IS_ERR(core->rstc))
+ return dev_err_probe(dev, PTR_ERR(core->rstc),
+ "cannot get resets\n");
+
+ if (iommu_get_domain_for_dev(dev)) {
+ core->empty_domain = iommu_paging_domain_alloc(dev);
+ if (IS_ERR(core->empty_domain)) {
+ dev_warn(dev, "cannot allocate empty IOMMU domain\n");
+ core->empty_domain = NULL;
+ } else {
+ ret = devm_add_action_or_reset(dev,
+ rkvenc_free_empty_domain,
+ core->empty_domain);
+ if (ret)
+ return ret;
+ }
+ }
+
+ ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ if (ret)
+ return dev_err_probe(dev, ret, "cannot set DMA mask\n");
+
+ vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32));
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ platform_set_drvdata(pdev, core);
+
+ pm_runtime_set_autosuspend_delay(dev, 100);
+ pm_runtime_use_autosuspend(dev);
+ ret = devm_pm_runtime_enable(dev);
+ if (ret)
+ return ret;
+
+ ret = devm_request_threaded_irq(dev, irq, NULL, rkvenc_irq_thread,
+ IRQF_ONESHOT, dev_name(dev), core);
+ if (ret)
+ return dev_err_probe(dev, ret, "cannot request IRQ\n");
+
+ ret = component_add(dev, &rkvenc_core_ops);
+ if (ret)
+ return dev_err_probe(dev, ret, "cannot add component\n");
+
+ return 0;
+}
+
+static void rkvenc_core_remove(struct platform_device *pdev)
+{
+ struct rkvenc_core *core = platform_get_drvdata(pdev);
+
+ component_del(&pdev->dev, &rkvenc_core_ops);
+ cancel_delayed_work_sync(&core->watchdog_work);
+}
+
+static int rkvenc_core_runtime_resume(struct device *dev)
+{
+ struct rkvenc_core *core = dev_get_drvdata(dev);
+
+ return clk_bulk_prepare_enable(core->num_clocks, core->clocks);
+}
+
+static int rkvenc_core_runtime_suspend(struct device *dev)
+{
+ struct rkvenc_core *core = dev_get_drvdata(dev);
+
+ clk_bulk_disable_unprepare(core->num_clocks, core->clocks);
+ return 0;
+}
+
+static const struct dev_pm_ops rkvenc_core_pm_ops = {
+ SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume)
+ RUNTIME_PM_OPS(rkvenc_core_runtime_suspend, rkvenc_core_runtime_resume,
+ NULL)
+};
+
+static const struct of_device_id rkvenc_core_of_match[] = {
+ { .compatible = "rockchip,rk3588-vepu580" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, rkvenc_core_of_match);
+
+static struct platform_driver rkvenc_core_driver = {
+ .probe = rkvenc_core_probe,
+ .remove = rkvenc_core_remove,
+ .driver = {
+ .name = RKVENC_NAME "-core",
+ .of_match_table = rkvenc_core_of_match,
+ .pm = pm_ptr(&rkvenc_core_pm_ops),
+ },
+};
+
+/* ---- master ---- */
+
+static int rkvenc_master_bind(struct device *dev)
+{
+ struct rkvenc_dev *rkvenc = dev_get_drvdata(dev);
+ int ret;
+
+ ret = component_bind_all(dev, rkvenc);
+ if (ret)
+ return ret;
+
+ if (!rkvenc->main_core) {
+ ret = -ENODEV;
+ goto err_unbind;
+ }
+
+ ret = rkvenc_v4l2_init(rkvenc);
+ if (ret)
+ goto err_unbind;
+
+ rkvenc->bound = true;
+ return 0;
+
+err_unbind:
+ component_unbind_all(dev, rkvenc);
+ rkvenc->main_core = NULL;
+ rkvenc->iommu_domain = NULL;
+ return ret;
+}
+
+static void rkvenc_master_unbind(struct device *dev)
+{
+ struct rkvenc_dev *rkvenc = dev_get_drvdata(dev);
+
+ /* Let running jobs finish before the cores go away. */
+ v4l2_m2m_suspend(rkvenc->m2m_dev);
+ rkvenc->bound = false;
+ rkvenc_v4l2_cleanup(rkvenc);
+ component_unbind_all(dev, rkvenc);
+ rkvenc->main_core = NULL;
+ rkvenc->iommu_domain = NULL;
+}
+
+static const struct component_master_ops rkvenc_master_ops = {
+ .bind = rkvenc_master_bind,
+ .unbind = rkvenc_master_unbind,
+};
+
+static int rkvenc_master_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct component_match *match = NULL;
+ struct device_node *np;
+ struct rkvenc_dev *rkvenc;
+ unsigned int num_cores = 0;
+
+ for_each_matching_node(np, rkvenc_core_of_match) {
+ if (!of_device_is_available(np))
+ continue;
+
+ component_match_add_release(dev, &match, component_release_of,
+ component_compare_of,
+ of_node_get(np));
+ num_cores++;
+ }
+
+ if (!match)
+ return -ENODEV;
+
+ if (num_cores > BITS_PER_LONG)
+ num_cores = BITS_PER_LONG;
+
+ rkvenc = devm_kzalloc(dev, sizeof(*rkvenc), GFP_KERNEL);
+ if (!rkvenc)
+ return -ENOMEM;
+
+ rkvenc->cores = devm_kcalloc(dev, num_cores, sizeof(*rkvenc->cores),
+ GFP_KERNEL);
+ if (!rkvenc->cores)
+ return -ENOMEM;
+
+ rkvenc->h264_scl = vepu580_gen_h264_scl(dev);
+ if (!rkvenc->h264_scl)
+ return -ENOMEM;
+
+ rkvenc->dev = dev;
+ rkvenc->num_cores = num_cores;
+ mutex_init(&rkvenc->vdev_lock);
+ spin_lock_init(&rkvenc->cores_lock);
+ dev_set_drvdata(dev, rkvenc);
+
+ return component_master_add_with_match(dev, &rkvenc_master_ops, match);
+}
+
+static void rkvenc_master_remove(struct platform_device *pdev)
+{
+ component_master_del(&pdev->dev, &rkvenc_master_ops);
+}
+
+/*
+ * The master device is registered after the cores, so it is suspended
+ * before and resumed after them: pause the job queue here so that no job
+ * is running when the cores are suspended.
+ */
+static int rkvenc_master_suspend(struct device *dev)
+{
+ struct rkvenc_dev *rkvenc = dev_get_drvdata(dev);
+
+ if (rkvenc && rkvenc->bound)
+ v4l2_m2m_suspend(rkvenc->m2m_dev);
+
+ return 0;
+}
+
+static int rkvenc_master_resume(struct device *dev)
+{
+ struct rkvenc_dev *rkvenc = dev_get_drvdata(dev);
+
+ if (rkvenc && rkvenc->bound)
+ v4l2_m2m_resume(rkvenc->m2m_dev);
+
+ return 0;
+}
+
+static DEFINE_SIMPLE_DEV_PM_OPS(rkvenc_master_pm_ops, rkvenc_master_suspend,
+ rkvenc_master_resume);
+
+static struct platform_driver rkvenc_master_driver = {
+ .probe = rkvenc_master_probe,
+ .remove = rkvenc_master_remove,
+ .driver = {
+ .name = RKVENC_NAME,
+ .pm = pm_sleep_ptr(&rkvenc_master_pm_ops),
+ },
+};
+
+static struct platform_device *rkvenc_master_pdev;
+
+static bool rkvenc_has_available_core(void)
+{
+ struct device_node *np;
+
+ for_each_matching_node(np, rkvenc_core_of_match) {
+ if (of_device_is_available(np)) {
+ of_node_put(np);
+ return true;
+ }
+ }
+
+ return false;
+}
+
+static int __init rkvenc_init(void)
+{
+ int ret;
+
+ ret = platform_driver_register(&rkvenc_core_driver);
+ if (ret)
+ return ret;
+
+ ret = platform_driver_register(&rkvenc_master_driver);
+ if (ret)
+ goto err_unregister_core;
+
+ if (!rkvenc_has_available_core())
+ return 0;
+
+ rkvenc_master_pdev = platform_device_register_simple(RKVENC_NAME,
+ PLATFORM_DEVID_NONE,
+ NULL, 0);
+ if (IS_ERR(rkvenc_master_pdev)) {
+ ret = PTR_ERR(rkvenc_master_pdev);
+ rkvenc_master_pdev = NULL;
+ goto err_unregister_master;
+ }
+
+ return 0;
+
+err_unregister_master:
+ platform_driver_unregister(&rkvenc_master_driver);
+err_unregister_core:
+ platform_driver_unregister(&rkvenc_core_driver);
+ return ret;
+}
+module_init(rkvenc_init);
+
+static void __exit rkvenc_exit(void)
+{
+ platform_device_unregister(rkvenc_master_pdev);
+ platform_driver_unregister(&rkvenc_master_driver);
+ platform_driver_unregister(&rkvenc_core_driver);
+}
+module_exit(rkvenc_exit);
+
+MODULE_DESCRIPTION("Rockchip VEPU580 stateless video encoder driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
new file mode 100644
index 000000000000..83a6084fc084
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
@@ -0,0 +1,236 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Rockchip VEPU580 (RKVENC) stateless video encoder driver
+ */
+
+#ifndef RKVENC_H_
+#define RKVENC_H_
+
+#include <linux/clk.h>
+#include <linux/platform_device.h>
+#include <linux/reset.h>
+#include <linux/spinlock.h>
+#include <linux/videodev2.h>
+#include <linux/workqueue.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-h264-enc.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "rkvenc-regs.h"
+
+struct rkvenc_ctx;
+struct rkvenc_core;
+
+/**
+ * struct rkvenc_aux_buf - driver-internal DMA buffer
+ * @cpu: kernel address, or NULL when allocated without a kernel mapping
+ * @dma: device address (IOVA in the shared IOMMU domain)
+ * @size: size in bytes
+ * @attrs: DMA attributes used for the allocation
+ */
+struct rkvenc_aux_buf {
+ void *cpu;
+ dma_addr_t dma;
+ size_t size;
+ unsigned long attrs;
+};
+
+/**
+ * struct rkvenc_rec_buf - reconstructed / reference picture
+ * @pixel: FBC header followed by the FBC body
+ * @thumb: 1/4 x 1/4 down-scaled luma used by the coarse motion search
+ * @colmv: collocated motion vectors (HEVC TMVP only)
+ * @fbc_hdr_size: offset of the FBC body inside @pixel
+ */
+struct rkvenc_rec_buf {
+ struct rkvenc_aux_buf pixel;
+ struct rkvenc_aux_buf thumb;
+ struct rkvenc_aux_buf colmv;
+ size_t fbc_hdr_size;
+};
+
+/**
+ * struct rkvenc_feedback - per-frame values read back from the status block
+ */
+struct rkvenc_feedback {
+ u32 int_sta;
+ u64 bs_len;
+ u64 sse;
+ u32 qp_sum;
+ u32 madi;
+ u32 madp;
+ u32 intra_blocks;
+ u32 cycles;
+};
+
+/**
+ * struct rkvenc_run - state of one hardware job
+ */
+struct rkvenc_run {
+ struct vb2_v4l2_buffer *src;
+ struct vb2_v4l2_buffer *dst;
+ /* Bytes written by software at the start of @dst (parameter sets). */
+ unsigned int hdr_bytes;
+};
+
+/**
+ * struct rkvenc_codec_ops - per-codec backend
+ * @adjust_fmt: adjust the coded (CAPTURE) format
+ * @start: called on STREAMON of both queues, allocates codec state
+ * @stop: called on STREAMOFF, frees codec state
+ * @run: prepare the job: produce software headers and fill @core's shadow
+ * registers (codec syntax, rate control and references)
+ * @done: called after the hardware finished; @fb holds the status read-back.
+ * The backend sets the CAPTURE payload and may change @state.
+ */
+struct rkvenc_codec_ops {
+ int (*adjust_fmt)(struct rkvenc_ctx *ctx, struct v4l2_format *f);
+ int (*start)(struct rkvenc_ctx *ctx);
+ void (*stop)(struct rkvenc_ctx *ctx);
+ int (*run)(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
+ struct rkvenc_run *run);
+ void (*done)(struct rkvenc_ctx *ctx, struct rkvenc_run *run,
+ const struct rkvenc_feedback *fb,
+ enum vb2_buffer_state *state);
+};
+
+struct rkvenc_ctrls {
+ const struct v4l2_ctrl_config *ctrls;
+ unsigned int num_ctrls;
+};
+
+/**
+ * struct rkvenc_coded_fmt_desc - coded (CAPTURE) format description
+ */
+struct rkvenc_coded_fmt_desc {
+ u32 fourcc;
+ struct v4l2_frmsize_stepwise frmsize;
+ const struct rkvenc_ctrls *ctrls;
+ const struct rkvenc_codec_ops *ops;
+};
+
+/**
+ * struct rkvenc_raw_fmt - raw (OUTPUT) format and its hardware mapping
+ */
+struct rkvenc_raw_fmt {
+ u32 fourcc;
+ u8 hw_fmt;
+ u8 rbuv_swap;
+};
+
+struct rkvenc_dev;
+
+/**
+ * struct rkvenc_core - one VEPU580 hardware core
+ */
+struct rkvenc_core {
+ struct device *dev;
+ struct rkvenc_dev *rkvenc;
+ void __iomem *regs;
+ struct clk_bulk_data *clocks;
+ int num_clocks;
+ struct clk *core_clk;
+ struct reset_control *rstc;
+ struct iommu_domain *empty_domain;
+ struct delayed_work watchdog_work;
+ /* Context whose job is running, protected by rkvenc->cores_lock. */
+ struct rkvenc_ctx *curr_ctx;
+ struct rkvenc_run run;
+ /* Shadow of the frame configuration registers. */
+ u32 *shadow;
+ unsigned long irq_count;
+ int id;
+};
+
+/**
+ * struct rkvenc_dev - V4L2 device, shared by all cores
+ */
+struct rkvenc_dev {
+ struct v4l2_device v4l2_dev;
+ struct media_device mdev;
+ struct video_device vdev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct device *dev;
+ /* Serializes ioctls. */
+ struct mutex vdev_lock;
+ struct rkvenc_core **cores;
+ unsigned int num_cores;
+ unsigned int core_count;
+ unsigned long idle_cores;
+ /* Protects idle_cores and core->curr_ctx. */
+ spinlock_t cores_lock;
+ struct rkvenc_core *main_core;
+ struct iommu_domain *iommu_domain;
+ /* The cores are bound and the V4L2 devices registered. */
+ bool bound;
+ /* H.264 flat quantisation tables, generated once. */
+ u32 *h264_scl;
+};
+
+/**
+ * struct rkvenc_h264_ctx - H.264 per-context state
+ * @enc: H.264 stateless encode core
+ * @hw_rc: the current picture uses the hardware row rate control
+ */
+struct rkvenc_h264_ctx {
+ struct v4l2_h264_enc enc;
+ bool hw_rc;
+};
+
+struct rkvenc_ctx {
+ struct v4l2_fh fh;
+ struct rkvenc_dev *dev;
+ struct v4l2_format coded_fmt;
+ struct v4l2_format src_fmt;
+ const struct rkvenc_coded_fmt_desc *coded_fmt_desc;
+ const struct rkvenc_raw_fmt *raw_fmt;
+ struct v4l2_fract timeperframe;
+ struct v4l2_ctrl_handler ctrl_hdl;
+ struct rkvenc_core *core;
+ struct v4l2_ctrl *avg_qp_ctrl;
+ void *priv;
+ bool started;
+};
+
+static inline struct rkvenc_ctx *file_to_rkvenc_ctx(struct file *filp)
+{
+ return container_of(file_to_v4l2_fh(filp), struct rkvenc_ctx, fh);
+}
+
+static inline void rkvenc_set(struct rkvenc_core *core, u32 reg, u32 val)
+{
+ core->shadow[reg / 4] = val;
+}
+
+static inline void rkvenc_or(struct rkvenc_core *core, u32 reg, u32 val)
+{
+ core->shadow[reg / 4] |= val;
+}
+
+/* Shared VEPU580 programming helpers (rkvenc-vepu580.c). */
+int rkvenc_alloc_aux(struct rkvenc_dev *rkvenc, struct rkvenc_aux_buf *buf,
+ size_t size, bool kernel_mapping);
+void rkvenc_free_aux(struct rkvenc_dev *rkvenc, struct rkvenc_aux_buf *buf);
+void vepu580_setup_common(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
+ struct rkvenc_run *run);
+void vepu580_setup_me(struct rkvenc_core *core, unsigned int width_mbs,
+ unsigned int height_mbs, unsigned int level_idc,
+ bool intra);
+void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
+ struct rkvenc_core *core, bool intra);
+void vepu580_setup_row_rc(struct rkvenc_core *core, unsigned int qp,
+ unsigned int qp_min, unsigned int qp_max,
+ unsigned int qp_range, u64 target_bits,
+ unsigned int num_blocks, unsigned int blocks_per_row);
+void vepu580_setup_fixed_qp(struct rkvenc_core *core, unsigned int qp);
+u32 *vepu580_gen_h264_scl(struct device *dev);
+dma_addr_t rkvenc_dst_addr(struct rkvenc_run *run);
+size_t rkvenc_dst_size(struct rkvenc_run *run);
+
+extern const struct rkvenc_coded_fmt_desc rkvenc_h264_fmt_desc;
+
+#endif /* RKVENC_H_ */
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 03/12] media: rockchip: rkvenc: Prepare the VEPU580 helpers for a second codec
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 01/12] dt-bindings: media: Add the Rockchip VEPU580 video encoder Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 02/12] media: rockchip: Add a VEPU580 stateless video encoder driver Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 04/12] media: rockchip: rkvenc: Add HEVC encoding Arun Iyengar via B4 Relay
` (8 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
Make the codec-independent VEPU580 programming usable by the coming HEVC
backend, without changing the H.264 programming:
- The encoded picture size alignment (enc_rsl and src_fill) comes from
the coded format: 16 for H.264, while HEVC uses 8.
- The hardware row rate control takes its codec-specific parameters (QP
adjustment per deviation bucket, outer threshold multiple, AQ mode)
from a table. The H.264 one holds the values used so far.
- The row_rc_qp_range module parameter moves to the shared code so that
both codecs honour it. Its name and default are unchanged.
- Controls shared by several coded formats are registered once.
- The status read-back also collects the inter and intra block counts
per block size and the lowest and highest QP of the 8x8 block QP
histogram, for the per-frame statistics.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
.../media/platform/rockchip/rkvenc/rkvenc-h264.c | 22 +++++-------
.../media/platform/rockchip/rkvenc/rkvenc-regs.h | 7 ++++
.../platform/rockchip/rkvenc/rkvenc-vepu580.c | 41 +++++++++++++++-------
drivers/media/platform/rockchip/rkvenc/rkvenc.c | 24 ++++++++++---
drivers/media/platform/rockchip/rkvenc/rkvenc.h | 41 +++++++++++++++++++++-
5 files changed, 103 insertions(+), 32 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
index c669a6322f29..227d341d060b 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
@@ -12,20 +12,12 @@
*/
#include <linux/bitfield.h>
-#include <linux/moduleparam.h>
#include <linux/sizes.h>
#include <linux/slab.h>
#include <media/v4l2-h264-enc.h>
#include "rkvenc.h"
-#define RKVENC_H264_ROW_QP_RANGE 2
-
-static unsigned int row_rc_qp_range = RKVENC_H264_ROW_QP_RANGE;
-module_param(row_rc_qp_range, uint, 0644);
-MODULE_PARM_DESC(row_rc_qp_range,
- "CBR: maximum per-row QP change by the hardware rate control (0 = frame-level rate control only)");
-
#define RKVENC_H264_PROFILE_BASELINE 66
#define RKVENC_H264_PROFILE_MAIN 77
#define RKVENC_H264_PROFILE_HIGH 100
@@ -390,15 +382,16 @@ static int rkvenc_h264_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
vepu580_setup_h264_tuning(ctx->dev, core, intra);
/*
- * In CBR mode the frame size target from the common rate control is
- * handed to the hardware row rate control, which adapts the QP of
- * each macroblock row by up to RKVENC_H264_ROW_QP_RANGE.
+ * In CBR mode the frame size target of the rate control is handed to
+ * the hardware row rate control, which adapts the QP of each
+ * macroblock row by up to the row_rc_qp_range parameter.
*/
h264->hw_rc = rc->mode && rc->mode->type == V4L2_H264_ENC_RC_TYPE_CBR &&
- READ_ONCE(row_rc_qp_range);
+ READ_ONCE(rkvenc_row_rc_qp_range);
if (h264->hw_rc)
- vepu580_setup_row_rc(core, rc->qp, state->qp_min, state->qp_max,
- min(READ_ONCE(row_rc_qp_range), 15U),
+ vepu580_setup_row_rc(core, &vepu580_h264_row_rc, rc->qp,
+ state->qp_min, state->qp_max,
+ min(READ_ONCE(rkvenc_row_rc_qp_range), 15U),
(u64)rc->size * 8,
mbs, state->width_mbs);
else
@@ -506,6 +499,7 @@ static const struct rkvenc_ctrls rkvenc_h264_ctrls = {
const struct rkvenc_coded_fmt_desc rkvenc_h264_fmt_desc = {
.fourcc = V4L2_PIX_FMT_H264_SLICE,
+ .size_align = 16,
.frmsize = {
.min_width = 64,
/* The external line buffer for wider pictures is not handled. */
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
index 6957cdf53446..8103bfdf7151 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
@@ -360,6 +360,11 @@
#define VEPU580_ST_BUS 0x4030
#define VEPU580_ST_SNUM 0x4034
#define VEPU580_ST_SLEN 0x4038
+#define VEPU580_ST_PNUM_P64 0x4100
+#define VEPU580_ST_PNUM_P32 0x4104
+#define VEPU580_ST_PNUM_P16 0x4108
+#define VEPU580_ST_PNUM_P8 0x410c
+#define VEPU580_ST_PNUM_I32 0x4110
#define VEPU580_ST_PNUM_I16 0x4114
#define VEPU580_ST_PNUM_I8 0x4118
#define VEPU580_ST_PNUM_I4 0x411c
@@ -367,6 +372,8 @@
#define VEPU580_ST_BNUM_CME 0x4124
#define VEPU580_ST_MADI 0x4128
#define VEPU580_ST_BNUM_B16 0x412c
+/* Histogram of the QP of the 8x8 blocks, one counter per QP 0..51 */
+#define VEPU580_ST_B8_QP(qp) (0x4200 + 4 * (qp))
/* Debug class */
#define VEPU580_DBG_FRAME_CYC 0x5200
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
index 33d5154ef8da..3a32caad2216 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
@@ -12,10 +12,16 @@
#include <linux/bitfield.h>
#include <linux/dma-mapping.h>
#include <linux/minmax.h>
+#include <linux/moduleparam.h>
#include <linux/slab.h>
#include "rkvenc.h"
+unsigned int rkvenc_row_rc_qp_range = 2;
+module_param_named(row_rc_qp_range, rkvenc_row_rc_qp_range, uint, 0644);
+MODULE_PARM_DESC(row_rc_qp_range,
+ "CBR: maximum per-row QP change by the hardware rate control (0 = frame-level rate control only)");
+
int rkvenc_alloc_aux(struct rkvenc_dev *rkvenc, struct rkvenc_aux_buf *buf,
size_t size, bool kernel_mapping)
{
@@ -74,17 +80,18 @@ static void vepu580_setup_src(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
struct vb2_buffer *vb = &run->src->vb2_buf;
unsigned int width = ctx->coded_fmt.fmt.pix_mp.width;
unsigned int height = ctx->coded_fmt.fmt.pix_mp.height;
+ unsigned int align = ctx->coded_fmt_desc->size_align;
unsigned int ystride = pix->plane_fmt[0].bytesperline;
unsigned int cstride = 0;
dma_addr_t addr[3] = { 0 };
unsigned int i;
rkvenc_set(core, VEPU580_ENC_RSL,
- FIELD_PREP(VEPU580_ENC_RSL_PIC_WD8_M1, ALIGN(width, 16) / 8 - 1) |
- FIELD_PREP(VEPU580_ENC_RSL_PIC_HD8_M1, ALIGN(height, 16) / 8 - 1));
+ FIELD_PREP(VEPU580_ENC_RSL_PIC_WD8_M1, ALIGN(width, align) / 8 - 1) |
+ FIELD_PREP(VEPU580_ENC_RSL_PIC_HD8_M1, ALIGN(height, align) / 8 - 1));
rkvenc_set(core, VEPU580_SRC_FILL,
- FIELD_PREP(VEPU580_SRC_FILL_PIC_WFILL, ALIGN(width, 16) - width) |
- FIELD_PREP(VEPU580_SRC_FILL_PIC_HFILL, ALIGN(height, 16) - height));
+ FIELD_PREP(VEPU580_SRC_FILL_PIC_WFILL, ALIGN(width, align) - width) |
+ FIELD_PREP(VEPU580_SRC_FILL_PIC_HFILL, ALIGN(height, align) - height));
/* MPP sets src_range for every YUV input; out_fmt selects 4:2:0. */
rkvenc_set(core, VEPU580_SRC_FMT,
@@ -450,17 +457,24 @@ void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
}
/*
- * Frame-level QP with the hardware row rate control: each macroblock row
- * gets target_bits / num_blocks bits per block; the cumulative deviation
- * from that target selects one of nine QP adjustments, bounded by
- * qp_range around the frame QP and by [qp_min, qp_max].
+ * Frame-level QP with the hardware row rate control: each block row
+ * (macroblock row for H.264, CTU row for HEVC) gets target_bits /
+ * num_blocks bits per block; the cumulative deviation from that target
+ * selects one of nine QP adjustments, bounded by qp_range around the frame
+ * QP and by [qp_min, qp_max].
*/
-void vepu580_setup_row_rc(struct rkvenc_core *core, unsigned int qp,
+const struct vepu580_row_rc vepu580_h264_row_rc = {
+ .qp_adj = { -2, -1, 0, 1, 2, 0, 0, 0, 1 },
+ .dthd_outer = 4,
+ .aq_mode = false,
+};
+
+void vepu580_setup_row_rc(struct rkvenc_core *core,
+ const struct vepu580_row_rc *cfg, unsigned int qp,
unsigned int qp_min, unsigned int qp_max,
unsigned int qp_range, u64 target_bits,
unsigned int num_blocks, unsigned int blocks_per_row)
{
- static const s8 qp_adj[9] = { -2, -1, 0, 1, 2, 0, 0, 0, 1 };
u64 ebit16 = div_u64(target_bits << 4, num_blocks);
s32 row_bits, thd;
unsigned int i;
@@ -474,6 +488,7 @@ void vepu580_setup_row_rc(struct rkvenc_core *core, unsigned int qp,
rkvenc_or(core, VEPU580_ENC_PIC, FIELD_PREP(VEPU580_ENC_PIC_PIC_QP, qp));
rkvenc_set(core, VEPU580_RC_CFG,
VEPU580_RC_CFG_RC_EN | VEPU580_RC_CFG_AQ_EN |
+ (cfg->aq_mode ? VEPU580_RC_CFG_AQ_MODE : 0) |
FIELD_PREP(VEPU580_RC_CFG_RC_CTU_NUM, blocks_per_row));
rkvenc_set(core, VEPU580_RC_QP,
FIELD_PREP(VEPU580_RC_QP_RANGE, qp_range) |
@@ -485,12 +500,12 @@ void vepu580_setup_row_rc(struct rkvenc_core *core, unsigned int qp,
rkvenc_set(core, VEPU580_RC_ADJ1, 0);
for (i = 0; i < 9; i++)
rkvenc_or(core, i < 5 ? VEPU580_RC_ADJ0 : VEPU580_RC_ADJ1,
- VEPU580_RC_ADJ_QP(i < 5 ? i : i - 5, qp_adj[i]));
+ VEPU580_RC_ADJ_QP(i < 5 ? i : i - 5, cfg->qp_adj[i]));
- rkvenc_set(core, VEPU580_RC_DTHD(0), -4 * thd);
+ rkvenc_set(core, VEPU580_RC_DTHD(0), -cfg->dthd_outer * thd);
rkvenc_set(core, VEPU580_RC_DTHD(1), -thd);
rkvenc_set(core, VEPU580_RC_DTHD(2), thd);
- rkvenc_set(core, VEPU580_RC_DTHD(3), 4 * thd);
+ rkvenc_set(core, VEPU580_RC_DTHD(3), cfg->dthd_outer * thd);
for (i = 4; i < 9; i++)
rkvenc_set(core, VEPU580_RC_DTHD(i), 0x7fffffff);
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
index 0e5afaefc0dd..623c50db25a8 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
@@ -720,6 +720,7 @@ static void rkvenc_read_feedback(struct rkvenc_core *core,
{
void __iomem *r = core->regs;
u32 sse_bsl = readl_relaxed(r + VEPU580_ST_SSE_BSL);
+ unsigned int i;
fb->bs_len = readl_relaxed(r + VEPU580_ST_BS_LGTH_L32) |
(u64)FIELD_GET(VEPU580_ST_BS_LGTH_H8, sse_bsl) << 32;
@@ -728,9 +729,20 @@ static void rkvenc_read_feedback(struct rkvenc_core *core,
fb->qp_sum = readl_relaxed(r + VEPU580_ST_QP_SUM);
fb->madi = readl_relaxed(r + VEPU580_ST_MADI);
fb->madp = readl_relaxed(r + VEPU580_ST_MADP);
- fb->intra_blocks = readl_relaxed(r + VEPU580_ST_PNUM_I16) +
- readl_relaxed(r + VEPU580_ST_PNUM_I8) +
- readl_relaxed(r + VEPU580_ST_PNUM_I4);
+ for (i = 0; i < RKVENC_PNUM_COUNT; i++)
+ fb->pnum[i] = readl_relaxed(r + VEPU580_ST_PNUM_P64 + 4 * i);
+ fb->intra_blocks = fb->pnum[RKVENC_PNUM_I16] + fb->pnum[RKVENC_PNUM_I8] +
+ fb->pnum[RKVENC_PNUM_I4];
+
+ fb->qp_min = -1;
+ fb->qp_max = -1;
+ for (i = 0; i < 52; i++) {
+ if (!readl_relaxed(r + VEPU580_ST_B8_QP(i)))
+ continue;
+ if (fb->qp_min < 0)
+ fb->qp_min = i;
+ fb->qp_max = i;
+ }
fb->cycles = readl_relaxed(r + VEPU580_DBG_FRAME_CYC);
}
@@ -941,8 +953,12 @@ static int rkvenc_init_ctrls(struct rkvenc_ctx *ctx)
for (i = 0; i < ARRAY_SIZE(rkvenc_coded_fmts); i++) {
const struct rkvenc_ctrls *ctrls = rkvenc_coded_fmts[i]->ctrls;
- for (j = 0; j < ctrls->num_ctrls; j++)
+ for (j = 0; j < ctrls->num_ctrls; j++) {
+ /* Controls shared by several codecs are added once. */
+ if (v4l2_ctrl_find(hdl, ctrls->ctrls[j].id))
+ continue;
v4l2_ctrl_new_custom(hdl, &ctrls->ctrls[j], NULL);
+ }
}
ctx->avg_qp_ctrl = v4l2_ctrl_find(hdl, V4L2_CID_MPEG_VIDEO_AVERAGE_QP);
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
index 83a6084fc084..5bc365101600 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
@@ -53,8 +53,24 @@ struct rkvenc_rec_buf {
size_t fbc_hdr_size;
};
+enum rkvenc_pnum {
+ RKVENC_PNUM_P64,
+ RKVENC_PNUM_P32,
+ RKVENC_PNUM_P16,
+ RKVENC_PNUM_P8,
+ RKVENC_PNUM_I32,
+ RKVENC_PNUM_I16,
+ RKVENC_PNUM_I8,
+ RKVENC_PNUM_I4,
+ RKVENC_PNUM_COUNT,
+};
+
/**
* struct rkvenc_feedback - per-frame values read back from the status block
+ * @pnum: number of inter (P64..P8) and intra (I32..I4) coded blocks per
+ * block size, see &enum rkvenc_pnum
+ * @qp_min: lowest QP of an 8x8 block, -1 when the histogram is empty
+ * @qp_max: highest QP of an 8x8 block, -1 when the histogram is empty
*/
struct rkvenc_feedback {
u32 int_sta;
@@ -64,6 +80,9 @@ struct rkvenc_feedback {
u32 madi;
u32 madp;
u32 intra_blocks;
+ u32 pnum[RKVENC_PNUM_COUNT];
+ int qp_min;
+ int qp_max;
u32 cycles;
};
@@ -105,9 +124,12 @@ struct rkvenc_ctrls {
/**
* struct rkvenc_coded_fmt_desc - coded (CAPTURE) format description
+ * @size_align: alignment of the encoded picture size (enc_rsl), the
+ * hardware pads the source up to it
*/
struct rkvenc_coded_fmt_desc {
u32 fourcc;
+ unsigned int size_align;
struct v4l2_frmsize_stepwise frmsize;
const struct rkvenc_ctrls *ctrls;
const struct rkvenc_codec_ops *ops;
@@ -222,7 +244,24 @@ void vepu580_setup_me(struct rkvenc_core *core, unsigned int width_mbs,
bool intra);
void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
struct rkvenc_core *core, bool intra);
-void vepu580_setup_row_rc(struct rkvenc_core *core, unsigned int qp,
+
+/**
+ * struct vepu580_row_rc - codec-specific row rate control parameters
+ * @qp_adj: QP adjustment for each of the nine deviation buckets
+ * @dthd_outer: multiple of the base threshold for the outer buckets
+ * @aq_mode: value of rc_cfg.aq_mode
+ */
+struct vepu580_row_rc {
+ s8 qp_adj[9];
+ unsigned int dthd_outer;
+ bool aq_mode;
+};
+
+extern const struct vepu580_row_rc vepu580_h264_row_rc;
+extern unsigned int rkvenc_row_rc_qp_range;
+
+void vepu580_setup_row_rc(struct rkvenc_core *core,
+ const struct vepu580_row_rc *cfg, unsigned int qp,
unsigned int qp_min, unsigned int qp_max,
unsigned int qp_range, u64 target_bits,
unsigned int num_blocks, unsigned int blocks_per_row);
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 04/12] media: rockchip: rkvenc: Add HEVC encoding
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (2 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 03/12] media: rockchip: rkvenc: Prepare the VEPU580 helpers for a second codec Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 05/12] media: rockchip: rkvenc: Don't enable adaptive QP with the row rate control Arun Iyengar via B4 Relay
` (7 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
Add an HEVC backend to the VEPU580 stateless encoder, on top of the V4L2
HEVC stateless encode core. The CAPTURE format is
V4L2_PIX_FMT_HEVC_SLICE, one access unit per buffer, Annex B.
The core writes the VPS, SPS and PPS at the start of the CAPTURE buffer;
the hardware writes the slice segment NAL unit and header from the HEVC
syntax registers, and the core patches nuh_temporal_id_plus1 afterwards,
which the hardware cannot write.
- Main profile, 8-bit 4:2:0, I and P pictures with one reference read
port (max_l0_refs = 1), single slice and tile, up to 4096x4096.
- The SPS is forced to the fixed coding structure of the hardware:
64x64 CTBs, 8x8 minimum CBs, 4x4 to 32x32 TBs with
max_transform_hierarchy_depth_inter/intra = 0 (MPP's TU depth 1 is
HM's QuadtreeTUMaxDepth, coded minus 1; found on the board), no AMP,
PCM, scaling lists, transform skip, transquant bypass,
constrained intra prediction, tiles or WPP. cu_qp_delta is always
enabled (as in MPP), so the row rate control and AQ can change the QP.
The hardware has one chroma QP offset, so Cr follows Cb.
- Inline short-term RPS of up to four negative pictures and up to three
long-term pictures, as the core resolves them; the reference picture
is the first entry of list 0. TMVP with collocated motion vector
buffers that are stored with each reference picture; motion vectors of
an intra collocated picture are not loaded. TMVP is not used with
long-term pictures.
- Recon/reference slots are FBC header + body, a 1/4 thumbnail and the
collocated MVs, sized per CTU with one extra CTU row.
- Fixed QP, constant quality and constant bitrate come from the core.
In the constant bitrate mode, the picture size target of the core
drives the hardware CTU row rate control (MPP's HEVC deviation
thresholds, row_rc_qp_range).
- Per-picture statistics (QP sum, count and range, luma SSE, intra and
inter block counts in 8x8 units) are passed to the core, which reports
them in V4L2_CID_STATELESS_HEVC_ENCODE_FEEDBACK. The request is
completed after that, as before.
- HEVC tuning: RDO thresholds and weights per CU size, segment and smear
thresholds, pre-intra and Sobel edge weights, intra cost thresholds,
CIME/RIME/FME settings, AQ steps, the quantisation dead zone and the
lambda tables (generated from their 2^(1/3) and 2^(1/6) progressions)
use the values of MPP's default configuration. The scaling list block
stays zero (scl_lst_sel = 0, flat).
The register layout and the default values were derived from the Rockchip
MPP library (hal_h265e_vepu580.c, hal_h265e_vepu580_reg.h,
hal_h265e_vepu580_tune.c, vepu5xx_common.c, h265e_ps.c, h265e_api.c,
Apache-2.0), read as documentation; no code was copied.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/platform/rockchip/rkvenc/Kconfig | 5 +-
drivers/media/platform/rockchip/rkvenc/Makefile | 1 +
.../media/platform/rockchip/rkvenc/rkvenc-hevc.c | 702 +++++++++++++++++++++
.../media/platform/rockchip/rkvenc/rkvenc-regs.h | 151 ++++-
.../platform/rockchip/rkvenc/rkvenc-vepu580.c | 343 +++++++++-
drivers/media/platform/rockchip/rkvenc/rkvenc.c | 5 +
drivers/media/platform/rockchip/rkvenc/rkvenc.h | 16 +
7 files changed, 1218 insertions(+), 5 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/Kconfig b/drivers/media/platform/rockchip/rkvenc/Kconfig
index 62bb1c7d5cc0..5398c4a77afa 100644
--- a/drivers/media/platform/rockchip/rkvenc/Kconfig
+++ b/drivers/media/platform/rockchip/rkvenc/Kconfig
@@ -8,10 +8,11 @@ config VIDEO_ROCKCHIP_VENC
select V4L2_MEM2MEM_DEV
select V4L2_H264
select V4L2_H264_ENC
+ select V4L2_HEVC_ENC
help
Support for the VEPU580 (RKVENC) video encoder of the Rockchip
- RK3588, using the V4L2 stateless encoder interface. H.264 is
- supported.
+ RK3588, using the V4L2 stateless encoder interface. H.264 and
+ HEVC are supported.
To compile this driver as a module, choose M here: the module
will be called rockchip-venc.
diff --git a/drivers/media/platform/rockchip/rkvenc/Makefile b/drivers/media/platform/rockchip/rkvenc/Makefile
index f41e2d2afdeb..d092543e339b 100644
--- a/drivers/media/platform/rockchip/rkvenc/Makefile
+++ b/drivers/media/platform/rockchip/rkvenc/Makefile
@@ -4,4 +4,5 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_VENC) += rockchip-venc.o
rockchip-venc-y += \
rkvenc.o \
rkvenc-h264.o \
+ rkvenc-hevc.o \
rkvenc-vepu580.o
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
new file mode 100644
index 000000000000..5bc6960a521a
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
@@ -0,0 +1,702 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip VEPU580 (RKVENC) HEVC encoding backend
+ *
+ * The video, sequence and picture parameter sets are generated by the V4L2
+ * HEVC stateless encode core and written by the CPU at the start of the
+ * CAPTURE buffer. The hardware writes the start code, NAL unit header and
+ * slice segment header from the syntax registers, followed by the slice
+ * data. It cannot write nuh_temporal_id_plus1, which the core patches
+ * after the run.
+ *
+ * The hardware reads exactly one reference picture per frame and has a
+ * fixed coding structure: 64x64 CTBs, 8x8 to 64x64 CBs, 4x4 to 32x32 TBs
+ * without explicit transform splits, no AMP, 8-bit 4:2:0. The SPS is forced
+ * to match it.
+ *
+ * Register meanings and default values come from the Rockchip MPP library
+ * (hal_h265e_vepu580.c, hal_h265e_vepu580_reg.h, h265e_ps.c).
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/log2.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+#include <media/v4l2-hevc-enc.h>
+
+#include "rkvenc.h"
+
+#define RKVENC_HEVC_CTB_SIZE 64
+/* max_mrg_cnd is a 2-bit field: MaxNumMergeCand 1..3 */
+#define RKVENC_HEVC_MAX_MERGE_CAND 3
+
+/* Inline short-term RPS: dlt_poc_s0_m10..m13, poc_lsb_lt0..2 */
+static const struct v4l2_hevc_enc_rps_limits rkvenc_hevc_rps_limits = {
+ .num_negative_pics = 4,
+ .num_positive_pics = 0,
+ .num_long_term_pics = 3,
+};
+
+/*
+ * Row rate control: MPP's HEVC deviation thresholds are {2n, n, p, 2p}
+ * with the adjustments {-2, -1, 0, 1, 2} and AQ mode 1.
+ */
+static const struct vepu580_row_rc rkvenc_hevc_row_rc = {
+ .qp_adj = { -2, -1, 0, 1, 2, 0, 0, 0, 0 },
+ .dthd_outer = 2,
+ .aq_mode = true,
+};
+
+static int rkvenc_hevc_state_constrain(struct v4l2_hevc_enc *enc,
+ struct v4l2_hevc_enc_state *state)
+{
+ struct rkvenc_ctx *ctx = enc->private_data;
+ const struct v4l2_pix_format_mplane *pix = &ctx->coded_fmt.fmt.pix_mp;
+ struct v4l2_ctrl_hevc_sps *sps = &state->sps;
+ struct v4l2_hevc_sps_video *video = &state->sps_video;
+ struct v4l2_ctrl_hevc_pps *pps = &state->pps;
+ struct v4l2_ctrl_hevc_encode_params *encode = &state->encode;
+ unsigned int coded_width = ALIGN(pix->width, 8);
+ unsigned int coded_height = ALIGN(pix->height, 8);
+
+ /* SPS: the coding structure of the hardware, Main profile. */
+ sps->bit_depth_luma_minus8 = 0;
+ sps->bit_depth_chroma_minus8 = 0;
+ sps->log2_min_luma_coding_block_size_minus3 = 0;
+ sps->log2_diff_max_min_luma_coding_block_size = 3;
+ sps->log2_min_luma_transform_block_size_minus2 = 0;
+ sps->log2_diff_max_min_luma_transform_block_size = 3;
+ /*
+ * MPP's "TU depth 1" is HM's QuadtreeTUMaxDepth, coded as depth - 1:
+ * the hardware never splits a transform tree beyond the implicit
+ * 64x64 -> 32x32 split (checked on the board, any other value makes
+ * the decoder misparse the residual).
+ */
+ sps->max_transform_hierarchy_depth_inter = 0;
+ sps->max_transform_hierarchy_depth_intra = 0;
+ sps->flags &= ~(V4L2_HEVC_SPS_FLAG_SEPARATE_COLOUR_PLANE |
+ V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED |
+ V4L2_HEVC_SPS_FLAG_AMP_ENABLED |
+ V4L2_HEVC_SPS_FLAG_PCM_ENABLED |
+ V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED);
+
+ /* Collocated long-term pictures are not handled, as in MPP. */
+ if (sps->flags & V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT) {
+ sps->flags &= ~V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED;
+ encode->flags &= ~V4L2_HEVC_ENCODE_FLAG_SLICE_TEMPORAL_MVP_ENABLED;
+ }
+
+ /*
+ * The core aligned the coded size to the minimum CB size it was given:
+ * redo it for 8x8 CBs, which is also the enc_rsl granularity.
+ */
+ sps->pic_width_in_luma_samples = coded_width;
+ sps->pic_height_in_luma_samples = coded_height;
+ state->ctb_log2_size = ilog2(RKVENC_HEVC_CTB_SIZE);
+ state->width_ctbs = DIV_ROUND_UP(coded_width, RKVENC_HEVC_CTB_SIZE);
+ state->height_ctbs = DIV_ROUND_UP(coded_height, RKVENC_HEVC_CTB_SIZE);
+
+ video->conf_win_left_offset = 0;
+ video->conf_win_top_offset = 0;
+ video->conf_win_right_offset = (coded_width - pix->width) / 2;
+ video->conf_win_bottom_offset = (coded_height - pix->height) / 2;
+ if (video->conf_win_right_offset || video->conf_win_bottom_offset)
+ video->flags |= V4L2_HEVC_SPS_VIDEO_FLAG_CONFORMANCE_WINDOW;
+ else
+ video->flags &= ~V4L2_HEVC_SPS_VIDEO_FLAG_CONFORMANCE_WINDOW;
+
+ /* PPS: tools without a register are off. */
+ pps->flags &= ~(V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED |
+ V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED |
+ V4L2_HEVC_PPS_FLAG_TRANSFORM_SKIP_ENABLED |
+ V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT |
+ V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED |
+ V4L2_HEVC_PPS_FLAG_TILES_ENABLED |
+ V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED |
+ V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED |
+ V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING |
+ V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT |
+ V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT);
+ pps->num_tile_columns_minus1 = 0;
+ pps->num_tile_rows_minus1 = 0;
+ pps->num_extra_slice_header_bits = 0;
+ pps->log2_parallel_merge_level_minus2 = 0;
+
+ /*
+ * The row rate control and adaptive quantisation change the QP within
+ * the picture: cu_qp_delta is always enabled, as MPP does.
+ */
+ pps->flags |= V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED;
+ pps->diff_cu_qp_delta_depth = min_t(u8, pps->diff_cu_qp_delta_depth, 2);
+
+ /* There is a single chroma QP offset register, for Cb and Cr. */
+ pps->pps_cb_qp_offset = clamp_t(s8, pps->pps_cb_qp_offset, -12, 12);
+ pps->pps_cr_qp_offset = pps->pps_cb_qp_offset;
+
+ /* Encode parameters */
+ encode->slice_cb_qp_offset = 0;
+ encode->slice_cr_qp_offset = 0;
+ if (encode->five_minus_max_num_merge_cand < 5 - RKVENC_HEVC_MAX_MERGE_CAND)
+ encode->five_minus_max_num_merge_cand =
+ 5 - RKVENC_HEVC_MAX_MERGE_CAND;
+ encode->flags &= ~V4L2_HEVC_ENCODE_FLAG_MVD_L1_ZERO;
+
+ return 0;
+}
+
+static int rkvenc_hevc_rec_buffer_alloc(struct v4l2_hevc_enc *enc,
+ struct v4l2_hevc_enc_rec_buffer *buffer)
+{
+ struct rkvenc_ctx *ctx = enc->private_data;
+ const struct v4l2_pix_format_mplane *pix = &ctx->coded_fmt.fmt.pix_mp;
+ /* FBC data needs one extra CTU row for the hardware I/O. */
+ size_t ctus = DIV_ROUND_UP(pix->width, RKVENC_HEVC_CTB_SIZE) *
+ (DIV_ROUND_UP(pix->height, RKVENC_HEVC_CTB_SIZE) + 1);
+ size_t hdr = ALIGN(ctus * 64, SZ_8K);
+ size_t body = ctus * 64 * 64 * 3 / 2;
+ struct rkvenc_rec_buf *rec;
+ int ret;
+
+ rec = kzalloc_obj(*rec);
+ if (!rec)
+ return -ENOMEM;
+
+ rec->fbc_hdr_size = hdr;
+
+ ret = rkvenc_alloc_aux(ctx->dev, &rec->pixel, hdr + body, false);
+ if (ret)
+ goto err_free;
+
+ ret = rkvenc_alloc_aux(ctx->dev, &rec->thumb, ALIGN(ctus * 256, SZ_4K),
+ false);
+ if (ret)
+ goto err_free_pixel;
+
+ ret = rkvenc_alloc_aux(ctx->dev, &rec->colmv, ALIGN(ctus * 96, SZ_4K),
+ false);
+ if (ret)
+ goto err_free_thumb;
+
+ buffer->private_data = rec;
+ return 0;
+
+err_free_thumb:
+ rkvenc_free_aux(ctx->dev, &rec->thumb);
+err_free_pixel:
+ rkvenc_free_aux(ctx->dev, &rec->pixel);
+err_free:
+ kfree(rec);
+ return ret;
+}
+
+static void rkvenc_hevc_rec_buffer_free(struct v4l2_hevc_enc *enc,
+ struct v4l2_hevc_enc_rec_buffer *buffer)
+{
+ struct rkvenc_ctx *ctx = enc->private_data;
+ struct rkvenc_rec_buf *rec = buffer->private_data;
+
+ if (!rec)
+ return;
+
+ rkvenc_free_aux(ctx->dev, &rec->colmv);
+ rkvenc_free_aux(ctx->dev, &rec->thumb);
+ rkvenc_free_aux(ctx->dev, &rec->pixel);
+ kfree(rec);
+ buffer->private_data = NULL;
+}
+
+static const struct v4l2_hevc_enc_ops rkvenc_hevc_enc_ops = {
+ .state_constrain = rkvenc_hevc_state_constrain,
+ .rec_buffer_alloc = rkvenc_hevc_rec_buffer_alloc,
+ .rec_buffer_free = rkvenc_hevc_rec_buffer_free,
+};
+
+/* ---- codec ops ---- */
+
+static int rkvenc_hevc_start(struct rkvenc_ctx *ctx)
+{
+ struct rkvenc_hevc_ctx *hevc;
+ struct v4l2_hevc_enc *enc;
+ int ret;
+
+ hevc = kzalloc_obj(*hevc);
+ if (!hevc)
+ return -ENOMEM;
+
+ enc = &hevc->enc;
+ enc->ops = &rkvenc_hevc_enc_ops;
+ enc->private_data = ctx;
+ enc->format_mplane = &ctx->coded_fmt.fmt.pix_mp;
+ enc->timeperframe = &ctx->timeperframe;
+ enc->ctrl_handler = &ctx->ctrl_hdl;
+ enc->ref_slots_count_init = 1;
+ enc->max_l0_refs = 1;
+ enc->max_l1_refs = 1;
+ enc->rps_limits = &rkvenc_hevc_rps_limits;
+ enc->flags = V4L2_HEVC_ENC_FLAG_INTER_PRED |
+ V4L2_HEVC_ENC_FLAG_HW_SLICE_HEADER;
+
+ ret = v4l2_hevc_enc_init(enc);
+ if (ret) {
+ kfree(hevc);
+ return ret;
+ }
+
+ ctx->priv = hevc;
+ return 0;
+}
+
+static void rkvenc_hevc_stop(struct rkvenc_ctx *ctx)
+{
+ struct rkvenc_hevc_ctx *hevc = ctx->priv;
+
+ if (!hevc)
+ return;
+
+ v4l2_hevc_enc_exit(&hevc->enc);
+ kfree(hevc);
+ ctx->priv = NULL;
+}
+
+static struct rkvenc_rec_buf *
+rkvenc_hevc_slot(struct v4l2_hevc_enc *enc, unsigned int index)
+{
+ return v4l2_hevc_enc_ref_buffer(enc, index)->private_data;
+}
+
+static bool rkvenc_hevc_ref_is_lt(struct v4l2_hevc_enc *enc)
+{
+ const struct v4l2_ctrl_hevc_encode_params *encode =
+ &enc->state_active.encode;
+ u8 idx;
+
+ if (!enc->ref.l0_count)
+ return false;
+
+ idx = enc->feedback.ref_idx_l0[0];
+ return idx < encode->num_refs &&
+ encode->refs[idx].flags & V4L2_HEVC_ENCODE_REF_FLAG_LONG_TERM;
+}
+
+/*
+ * The slice flags the active SPS allows: the core checks them against the
+ * SPS of the controls, which only becomes active at the next IRAP picture.
+ */
+static u64 rkvenc_hevc_slice_flags(const struct v4l2_hevc_enc_state *state,
+ const struct v4l2_hevc_slice_header *sh)
+{
+ u64 flags = sh->flags;
+
+ if (!(state->sps.flags & V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET))
+ flags &= ~(V4L2_HEVC_ENCODE_FLAG_SLICE_SAO_LUMA |
+ V4L2_HEVC_ENCODE_FLAG_SLICE_SAO_CHROMA);
+ if (!(state->sps.flags & V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED))
+ flags &= ~V4L2_HEVC_ENCODE_FLAG_SLICE_TEMPORAL_MVP_ENABLED;
+
+ return flags;
+}
+
+static void rkvenc_hevc_setup_syntax(struct v4l2_hevc_enc *enc,
+ struct rkvenc_core *core)
+{
+ const struct v4l2_hevc_enc_state *state = &enc->state_active;
+ const struct v4l2_ctrl_hevc_sps *sps = &state->sps;
+ const struct v4l2_ctrl_hevc_pps *pps = &state->pps;
+ const struct v4l2_hevc_slice_header *sh = &enc->slice;
+ const struct v4l2_hevc_st_rps *st = &sh->st_rps;
+ u64 flags = rkvenc_hevc_slice_flags(state, sh);
+ unsigned int num_lt = min_t(unsigned int, sh->num_long_term_pics, 3);
+ unsigned int tot_cur, i;
+ bool dblk_dis;
+ s8 beta, tc;
+ u32 val;
+
+ tot_cur = hweight16(st->used_by_curr_pic_s0) +
+ hweight16(st->used_by_curr_pic_s1);
+ for (i = 0; i < num_lt; i++)
+ tot_cur += sh->lt[i].used_by_curr_pic_lt;
+
+ rkvenc_or(core, VEPU580_ENC_PIC,
+ FIELD_PREP(VEPU580_ENC_PIC_ENC_STND, VEPU580_ENC_STND_HEVC) |
+ ((state->encode.flags & V4L2_HEVC_ENCODE_FLAG_REFERENCE) ?
+ VEPU580_ENC_PIC_CUR_FRM_REF : 0) |
+ VEPU580_ENC_PIC_BS_SCP |
+ FIELD_PREP(VEPU580_ENC_PIC_NUM_PIC_TOT_CUR, tot_cur) |
+ FIELD_PREP(VEPU580_ENC_PIC_LOG2_CTU_NUM,
+ order_base_2(state->width_ctbs * state->height_ctbs)));
+
+ rkvenc_set(core, VEPU580_HEVC_SYNT_NAL,
+ FIELD_PREP(VEPU580_HEVC_NAL_UNIT_TYPE, sh->nal_unit_type));
+
+ val = FIELD_PREP(VEPU580_HEVC_SPS_LOG2_MAX_POC_LSB,
+ sps->log2_max_pic_order_cnt_lsb_minus4);
+ if (sps->flags & V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET)
+ val |= VEPU580_HEVC_SPS_SAO_EN;
+ if (sps->flags & V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT)
+ val |= VEPU580_HEVC_SPS_LT_REF_PIC_PRSNT;
+ if (sps->flags & V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED)
+ val |= VEPU580_HEVC_SPS_TMVP_EN;
+ if (sps->flags & V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED)
+ val |= VEPU580_HEVC_SPS_STRG_INTRA_SMTH;
+ rkvenc_set(core, VEPU580_HEVC_SYNT_SPS, val);
+
+ val = FIELD_PREP(VEPU580_HEVC_PPS_PIC_INIT_QP, pps->init_qp_minus26 + 26) |
+ FIELD_PREP(VEPU580_HEVC_PPS_CU_QP_DLT_DEPTH,
+ pps->diff_cu_qp_delta_depth);
+ if (pps->flags & V4L2_HEVC_PPS_FLAG_OUTPUT_FLAG_PRESENT)
+ val |= VEPU580_HEVC_PPS_OUT_FLG_PRSNT;
+ if (pps->flags & V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED)
+ val |= VEPU580_HEVC_PPS_SGN_DAT_HID_EN;
+ if (pps->flags & V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT)
+ val |= VEPU580_HEVC_PPS_CBC_INIT_PRSNT;
+ if (pps->flags & V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED)
+ val |= VEPU580_HEVC_PPS_CU_QP_DLT_EN;
+ if (pps->flags & V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED)
+ val |= VEPU580_HEVC_PPS_LP_FLTR_ACRS_SLI;
+ if (pps->flags & V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED)
+ val |= VEPU580_HEVC_PPS_DBLK_FLTR_OVRD_EN;
+ rkvenc_set(core, VEPU580_HEVC_SYNT_PPS, val);
+
+ val = FIELD_PREP(VEPU580_HEVC_SLI0_SLI_TYPE, sh->slice_type) |
+ FIELD_PREP(VEPU580_HEVC_SLI0_SLI_PPS_ID, pps->pic_parameter_set_id) |
+ FIELD_PREP(VEPU580_HEVC_SLI0_NUM_REFIDX_L0_ACT,
+ sh->num_ref_idx_l0_active_minus1) |
+ VEPU580_HEVC_SLI0_MRG_UP_FLG | VEPU580_HEVC_SLI0_MRG_LFT_FLG;
+ if (flags & V4L2_HEVC_ENCODE_FLAG_CABAC_INIT)
+ val |= VEPU580_HEVC_SLI0_CBC_INIT_FLG;
+ if (flags & V4L2_HEVC_ENCODE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE)
+ val |= VEPU580_HEVC_SLI0_NUM_REFIDX_ACT_OVRD;
+ if (flags & V4L2_HEVC_ENCODE_FLAG_SLICE_SAO_CHROMA)
+ val |= VEPU580_HEVC_SLI0_SAO_CHRM_FLG;
+ if (flags & V4L2_HEVC_ENCODE_FLAG_SLICE_SAO_LUMA)
+ val |= VEPU580_HEVC_SLI0_SAO_LUMA_FLG;
+ if (flags & V4L2_HEVC_ENCODE_FLAG_SLICE_TEMPORAL_MVP_ENABLED)
+ val |= VEPU580_HEVC_SLI0_TMPRL_MVP_E;
+ if (!(flags & V4L2_HEVC_ENCODE_FLAG_NO_PIC_OUTPUT))
+ val |= VEPU580_HEVC_SLI0_PIC_OUT_FLG;
+ if (flags & V4L2_HEVC_ENCODE_FLAG_NO_OUTPUT_OF_PRIOR_PICS)
+ val |= VEPU580_HEVC_SLI0_NO_OUT_PRI_PIC;
+ rkvenc_set(core, VEPU580_HEVC_SYNT_SLI0, val);
+
+ /*
+ * The deblocking registers hold the values in effect, which are the
+ * PPS ones when the slice does not override them.
+ */
+ if (flags & V4L2_HEVC_ENCODE_FLAG_DEBLOCKING_FILTER_OVERRIDE) {
+ dblk_dis = flags & V4L2_HEVC_ENCODE_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED;
+ beta = sh->slice_beta_offset_div2;
+ tc = sh->slice_tc_offset_div2;
+ } else {
+ dblk_dis = pps->flags & V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER;
+ beta = pps->pps_beta_offset_div2;
+ tc = pps->pps_tc_offset_div2;
+ }
+
+ val = FIELD_PREP(VEPU580_HEVC_SLI1_TC_OFST_DIV2, tc & 0xf) |
+ FIELD_PREP(VEPU580_HEVC_SLI1_BETA_OFST_DIV2, beta & 0xf) |
+ FIELD_PREP(VEPU580_HEVC_SLI1_CB_QP_OFST, pps->pps_cb_qp_offset & 0x1f) |
+ FIELD_PREP(VEPU580_HEVC_SLI1_SLI_QP, enc->qp) |
+ FIELD_PREP(VEPU580_HEVC_SLI1_MAX_MRG_CND,
+ 5 - sh->five_minus_max_num_merge_cand) |
+ FIELD_PREP(VEPU580_HEVC_SLI1_COL_REF_IDX, sh->collocated_ref_idx);
+ if (flags & V4L2_HEVC_ENCODE_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED)
+ val |= VEPU580_HEVC_SLI1_LP_FLTR_ACRS_SLI;
+ if (dblk_dis)
+ val |= VEPU580_HEVC_SLI1_DBLK_FLTR_DIS;
+ if (flags & V4L2_HEVC_ENCODE_FLAG_DEBLOCKING_FILTER_OVERRIDE)
+ val |= VEPU580_HEVC_SLI1_DBLK_FLTR_OVRD_FLG;
+ if (sh->slice_type != V4L2_HEVC_SLICE_TYPE_B ||
+ flags & V4L2_HEVC_ENCODE_FLAG_COLLOCATED_FROM_L0)
+ val |= VEPU580_HEVC_SLI1_COL_FRM_L0_FLG;
+ rkvenc_set(core, VEPU580_HEVC_SYNT_SLI1, val);
+
+ rkvenc_set(core, VEPU580_HEVC_SYNT_SLI2,
+ FIELD_PREP(VEPU580_HEVC_SLI2_POC_LSB,
+ sh->slice_pic_order_cnt_lsb));
+
+ /* Inline short-term RPS (st_ref_pic_set_sps_flag = 0) and LT pictures */
+ rkvenc_set(core, VEPU580_HEVC_SYNT_REFM0,
+ FIELD_PREP(VEPU580_HEVC_REFM0_POC_LSB_LT0,
+ num_lt > 0 ? sh->lt[0].poc_lsb_lt : 0) |
+ FIELD_PREP(VEPU580_HEVC_REFM0_NUM_LT_PIC, num_lt));
+
+ val = FIELD_PREP(VEPU580_HEVC_REFM1_USED_BY_S0,
+ st->used_by_curr_pic_s0) |
+ FIELD_PREP(VEPU580_HEVC_REFM1_NUM_NEG_PICS, st->num_negative_pics);
+ if (num_lt > 0)
+ val |= FIELD_PREP(VEPU580_HEVC_REFM1_DLT_POC_MSB_CYCL0,
+ sh->lt[0].delta_poc_msb_cycle_lt);
+ for (i = 0; i < num_lt; i++) {
+ if (sh->lt[i].delta_poc_msb_present)
+ val |= VEPU580_HEVC_REFM1_DLT_POC_MSB_PRSNT(i);
+ if (sh->lt[i].used_by_curr_pic_lt)
+ val |= VEPU580_HEVC_REFM1_USED_BY_LT(i);
+ }
+ rkvenc_set(core, VEPU580_HEVC_SYNT_REFM1, val);
+
+ rkvenc_set(core, VEPU580_HEVC_SYNT_REFM2,
+ st->delta_poc_s0_minus1[0] | st->delta_poc_s0_minus1[1] << 16);
+ rkvenc_set(core, VEPU580_HEVC_SYNT_REFM3,
+ st->delta_poc_s0_minus1[2] | st->delta_poc_s0_minus1[3] << 16);
+ rkvenc_set(core, VEPU580_HEVC_SYNT_LONG_REFM0,
+ FIELD_PREP(VEPU580_HEVC_LONG_REFM0_POC_LSB_LT1,
+ num_lt > 1 ? sh->lt[1].poc_lsb_lt : 0) |
+ FIELD_PREP(VEPU580_HEVC_LONG_REFM0_POC_LSB_LT2,
+ num_lt > 2 ? sh->lt[2].poc_lsb_lt : 0));
+ rkvenc_set(core, VEPU580_HEVC_SYNT_LONG_REFM1,
+ FIELD_PREP(VEPU580_HEVC_LONG_REFM1_MSB_CYCL1,
+ num_lt > 1 ? sh->lt[1].delta_poc_msb_cycle_lt : 0) |
+ FIELD_PREP(VEPU580_HEVC_LONG_REFM1_MSB_CYCL2,
+ num_lt > 2 ? sh->lt[2].delta_poc_msb_cycle_lt : 0));
+
+ /* RDO: MPP's CU and intra mode enables */
+ rkvenc_set(core, VEPU580_RDO_CFG,
+ (rkvenc_hevc_ref_is_lt(enc) ? VEPU580_HEVC_RDO_CFG_LTM_IDX0L0 : 0) |
+ VEPU580_HEVC_RDO_CFG_CHRM_SPCL |
+ FIELD_PREP(VEPU580_HEVC_RDO_CFG_CU_INTER_E, 0x6db) |
+ FIELD_PREP(VEPU580_HEVC_RDO_CFG_CU_INTRA_E, 0xf) |
+ VEPU580_HEVC_RDO_CFG_CCWA_E);
+
+ /* 0x3f0 is tile_cfg for HEVC: a single tile. */
+ rkvenc_set(core, VEPU580_HEVC_TILE_CFG, 0);
+ rkvenc_set(core, VEPU580_TILE_POS, 0);
+}
+
+static int rkvenc_hevc_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
+ struct rkvenc_run *run)
+{
+ struct rkvenc_hevc_ctx *hevc = ctx->priv;
+ struct v4l2_hevc_enc *enc = &hevc->enc;
+ const struct v4l2_hevc_enc_state *state = &enc->state_active;
+ struct rkvenc_rec_buf *rec, *refr = NULL, *col = NULL;
+ unsigned int ctus, qp_range;
+ bool intra, tmvp, colmv_stor;
+ int ret;
+
+ /* The parameter sets are written through the kernel mapping. */
+ if (!vb2_plane_vaddr(&run->dst->vb2_buf, 0))
+ return -ENOMEM;
+
+ ret = v4l2_hevc_enc_step(enc, run->dst);
+ if (ret)
+ return ret;
+
+ run->hdr_bytes = enc->header_size;
+ if (run->hdr_bytes + SZ_4K > rkvenc_dst_size(run))
+ return -ENOSPC;
+
+ intra = state->encode.slice_type == V4L2_HEVC_SLICE_TYPE_I;
+ ctus = state->width_ctbs * state->height_ctbs;
+
+ rec = enc->ref.buffer_current.private_data;
+ if (!rec)
+ return -EINVAL;
+
+ /* A P picture must have a reconstructed reference to read. */
+ if (!intra) {
+ if (!enc->ref.l0_count)
+ return -EINVAL;
+ refr = rkvenc_hevc_slot(enc, enc->ref.l0[0]);
+ if (!refr)
+ return -EINVAL;
+ }
+
+ tmvp = rkvenc_hevc_slice_flags(state, &enc->slice) &
+ V4L2_HEVC_ENCODE_FLAG_SLICE_TEMPORAL_MVP_ENABLED;
+ if (tmvp && enc->ref.collocated >= 0)
+ col = rkvenc_hevc_slot(enc, enc->ref.collocated);
+
+ /*
+ * As in MPP, only inter pictures store their motion vectors; those of
+ * an intra collocated picture are not loaded (they are unavailable).
+ */
+ colmv_stor = !intra &&
+ state->sps.flags & V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED;
+ rec->colmv_valid = colmv_stor;
+
+ vepu580_setup_common(ctx, core, run);
+
+ rkvenc_set(core, VEPU580_DTRNS_CFG, intra ? 0 : VEPU580_DTRNS_CFG_DSPR_OTSD);
+ rkvenc_set(core, VEPU580_SRC_PROC, VEPU580_SRC_PROC_TXA_EN);
+
+ rkvenc_hevc_setup_syntax(enc, core);
+
+ /* Reconstruction (written only for reference pictures) and reference */
+ rkvenc_set(core, VEPU580_RFPW_H_ADDR, rec->pixel.dma);
+ rkvenc_set(core, VEPU580_RFPW_B_ADDR, rec->pixel.dma + rec->fbc_hdr_size);
+ rkvenc_set(core, VEPU580_DSPW_ADDR, rec->thumb.dma);
+ rkvenc_set(core, VEPU580_CMVW_ADDR, rec->colmv.dma);
+ if (refr) {
+ rkvenc_set(core, VEPU580_RFPR_H_ADDR, refr->pixel.dma);
+ rkvenc_set(core, VEPU580_RFPR_B_ADDR,
+ refr->pixel.dma + refr->fbc_hdr_size);
+ rkvenc_set(core, VEPU580_DSPR_ADDR, refr->thumb.dma);
+ }
+ if (col)
+ rkvenc_set(core, VEPU580_CMVR_ADDR, col->colmv.dma);
+
+ vepu580_setup_hevc_me(core, state->sps.pic_width_in_luma_samples,
+ state->sps.pic_height_in_luma_samples,
+ colmv_stor, col && col->colmv_valid);
+ vepu580_setup_hevc_tuning(core, intra);
+
+ /*
+ * In CBR mode the size target of the frame rate control is handed to
+ * the hardware CTU row rate control, which adapts the QP of each CTU
+ * row by up to the row_rc_qp_range parameter.
+ */
+ qp_range = min(READ_ONCE(rkvenc_row_rc_qp_range), 15U);
+ hevc->hw_rc = state->frame_rc_enable &&
+ state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR &&
+ qp_range;
+ if (hevc->hw_rc)
+ vepu580_setup_row_rc(core, &rkvenc_hevc_row_rc, enc->qp,
+ clamp(state->qp_min, 0, 51),
+ clamp(state->qp_max, 0, 51), qp_range,
+ enc->cbr.target_bits, ctus,
+ state->width_ctbs);
+ else
+ vepu580_setup_fixed_qp(core, enc->qp);
+
+ return 0;
+}
+
+static void rkvenc_hevc_done(struct rkvenc_ctx *ctx, struct rkvenc_run *run,
+ const struct rkvenc_feedback *fb,
+ enum vb2_buffer_state *state)
+{
+ struct rkvenc_hevc_ctx *hevc = ctx->priv;
+ struct v4l2_stateless_encode_stats stats = { };
+ const u32 *pnum = fb->pnum;
+ struct v4l2_hevc_enc *enc;
+ unsigned int blocks;
+ u64 bytes;
+
+ if (!hevc || *state != VB2_BUF_STATE_DONE)
+ return;
+
+ enc = &hevc->enc;
+ bytes = run->hdr_bytes + fb->bs_len;
+ if (!fb->bs_len || bytes > rkvenc_dst_size(run)) {
+ *state = VB2_BUF_STATE_ERROR;
+ return;
+ }
+
+ vb2_set_plane_payload(&run->dst->vb2_buf, 0, bytes);
+
+ /* qp_sum counts the 8x8 blocks of the CTU-aligned picture. */
+ blocks = enc->state_active.width_ctbs * enc->state_active.height_ctbs *
+ (RKVENC_HEVC_CTB_SIZE / 8) * (RKVENC_HEVC_CTB_SIZE / 8);
+
+ stats.flags = V4L2_STATELESS_ENCODE_STATS_FLAG_QP |
+ V4L2_STATELESS_ENCODE_STATS_FLAG_SSE_LUMA |
+ V4L2_STATELESS_ENCODE_STATS_FLAG_BLOCKS;
+ stats.qp_sum = fb->qp_sum;
+ stats.qp_count = blocks;
+ if (fb->qp_min >= 0) {
+ stats.qp_min = fb->qp_min;
+ stats.qp_max = fb->qp_max;
+ } else {
+ stats.qp_min = enc->qp;
+ stats.qp_max = enc->qp;
+ }
+ stats.sse_luma = fb->sse;
+ /* In 8x8 units; I8 and I4 both count 8x8 CUs. */
+ stats.num_intra_blocks = pnum[RKVENC_PNUM_I32] * 16 +
+ pnum[RKVENC_PNUM_I16] * 4 +
+ pnum[RKVENC_PNUM_I8] + pnum[RKVENC_PNUM_I4];
+ stats.num_inter_blocks = pnum[RKVENC_PNUM_P64] * 64 +
+ pnum[RKVENC_PNUM_P32] * 16 +
+ pnum[RKVENC_PNUM_P16] * 4 + pnum[RKVENC_PNUM_P8];
+
+ if (ctx->avg_qp_ctrl && blocks)
+ v4l2_ctrl_s_ctrl(ctx->avg_qp_ctrl,
+ min(DIV_ROUND_CLOSEST(fb->qp_sum, blocks), 51U));
+
+ /* DPB update, TemporalId patch, buffer flags and the feedback control */
+ if (v4l2_hevc_enc_complete(enc, run->dst, &stats))
+ *state = VB2_BUF_STATE_ERROR;
+}
+
+static int rkvenc_hevc_adjust_fmt(struct rkvenc_ctx *ctx, struct v4l2_format *f)
+{
+ return 0;
+}
+
+static const struct rkvenc_codec_ops rkvenc_hevc_ops = {
+ .adjust_fmt = rkvenc_hevc_adjust_fmt,
+ .start = rkvenc_hevc_start,
+ .stop = rkvenc_hevc_stop,
+ .run = rkvenc_hevc_run,
+ .done = rkvenc_hevc_done,
+};
+
+static const struct v4l2_ctrl_config rkvenc_hevc_ctrl_cfgs[] = {
+ { .id = V4L2_CID_STATELESS_HEVC_SPS, },
+ { .id = V4L2_CID_STATELESS_HEVC_PPS, },
+ { .id = V4L2_CID_STATELESS_HEVC_ENCODE_PARAMS, },
+ /* Read-only, set by the core when a picture completes */
+ { .id = V4L2_CID_STATELESS_HEVC_ENCODE_FEEDBACK, },
+ {
+ .id = V4L2_CID_STATELESS_HEVC_START_CODE,
+ .min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+ .max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+ .def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+ .min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+ .max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+ .def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_HEVC_TIER,
+ .max = V4L2_MPEG_VIDEO_HEVC_TIER_HIGH,
+ .def = V4L2_MPEG_VIDEO_HEVC_TIER_MAIN,
+ },
+ { .id = V4L2_CID_MPEG_VIDEO_AU_DELIMITER, .max = 1, .step = 1, },
+ { .id = V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR, .max = 1, .step = 1, },
+ { .id = V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE, .max = 1, .step = 1, },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
+ .min = V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
+ .max = V4L2_MPEG_VIDEO_BITRATE_MODE_CQ,
+ .def = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
+ .menu_skip_mask = BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_VBR),
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE,
+ .min = 10000, .max = 200000000, .step = 1, .def = 10000000,
+ },
+ {
+ .id = V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY,
+ .min = 1, .max = 100, .step = 1, .def = 50,
+ },
+ { .id = V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP, .max = 51, .step = 1, .def = 10, },
+ { .id = V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP, .max = 51, .step = 1, .def = 51, },
+ { .id = V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP, .max = 51, .step = 1, .def = 30, },
+ { .id = V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP, .max = 51, .step = 1, .def = 32, },
+ { .id = V4L2_CID_MPEG_VIDEO_AVERAGE_QP, .max = 51, .step = 1, },
+};
+
+static const struct rkvenc_ctrls rkvenc_hevc_ctrls = {
+ .ctrls = rkvenc_hevc_ctrl_cfgs,
+ .num_ctrls = ARRAY_SIZE(rkvenc_hevc_ctrl_cfgs),
+};
+
+const struct rkvenc_coded_fmt_desc rkvenc_hevc_fmt_desc = {
+ .fourcc = V4L2_PIX_FMT_HEVC_SLICE,
+ .size_align = 8,
+ .frmsize = {
+ .min_width = 64,
+ /* Wider pictures need tiles, which are not implemented. */
+ .max_width = 4096,
+ .step_width = 2,
+ .min_height = 64,
+ .max_height = 4096,
+ .step_height = 2,
+ },
+ .ctrls = &rkvenc_hevc_ctrls,
+ .ops = &rkvenc_hevc_ops,
+};
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
index 8103bfdf7151..e8e7ce045f90 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
@@ -4,7 +4,8 @@
*
* The register meanings and field layouts were taken from the Rockchip MPP
* user space library register structures (vepu580_common.h,
- * hal_h264e_vepu580_reg.h). No TRM chapter was available.
+ * hal_h264e_vepu580_reg.h, hal_h265e_vepu580_reg.h). No TRM chapter was
+ * available.
*/
#ifndef RKVENC_REGS_H_
@@ -248,6 +249,106 @@
#define VEPU580_H264_REFM0_ARPM_FLG BIT(2)
#define VEPU580_H264_SYNT_REFM1 0x03cc
#define VEPU580_H264_SYNT_REFM2 0x03d0
+/* Frame class: HEVC variants of the shared registers */
+#define VEPU580_HEVC_ME_CFG_RME_DIS GENMASK(26, 23)
+#define VEPU580_HEVC_ME_CFG_FME_DIS GENMASK(31, 28)
+#define VEPU580_HEVC_RDO_CFG_LTM_COL BIT(0)
+#define VEPU580_HEVC_RDO_CFG_LTM_IDX0L0 BIT(1)
+#define VEPU580_HEVC_RDO_CFG_CHRM_SPCL BIT(2)
+#define VEPU580_HEVC_RDO_CFG_CU_INTER_E GENMASK(14, 3)
+#define VEPU580_HEVC_RDO_CFG_CU_INTRA_E GENMASK(22, 19)
+#define VEPU580_HEVC_RDO_CFG_CCWA_E BIT(23)
+#define VEPU580_HEVC_RDO_CFG_SCL_LST_SEL GENMASK(25, 24)
+#define VEPU580_HEVC_RDO_CFG_SATD_BYPS GENMASK(31, 28)
+
+/* Frame class: HEVC syntax */
+#define VEPU580_HEVC_SYNT_NAL 0x03b0
+#define VEPU580_HEVC_NAL_UNIT_TYPE GENMASK(5, 0)
+#define VEPU580_HEVC_SYNT_SPS 0x03b4
+#define VEPU580_HEVC_SPS_SAO_EN BIT(0)
+#define VEPU580_HEVC_SPS_NUM_ST_REF_PIC GENMASK(7, 1)
+#define VEPU580_HEVC_SPS_LT_REF_PIC_PRSNT BIT(8)
+#define VEPU580_HEVC_SPS_NUM_LT_REF_PIC GENMASK(14, 9)
+#define VEPU580_HEVC_SPS_TMVP_EN BIT(15)
+#define VEPU580_HEVC_SPS_LOG2_MAX_POC_LSB GENMASK(19, 16)
+#define VEPU580_HEVC_SPS_STRG_INTRA_SMTH BIT(20)
+#define VEPU580_HEVC_SYNT_PPS 0x03b8
+#define VEPU580_HEVC_PPS_DPDNT_SLI_SEG_EN BIT(0)
+#define VEPU580_HEVC_PPS_OUT_FLG_PRSNT BIT(1)
+#define VEPU580_HEVC_PPS_NUM_EXTR_SLI_HDR GENMASK(4, 2)
+#define VEPU580_HEVC_PPS_SGN_DAT_HID_EN BIT(5)
+#define VEPU580_HEVC_PPS_CBC_INIT_PRSNT BIT(6)
+#define VEPU580_HEVC_PPS_PIC_INIT_QP GENMASK(12, 7)
+#define VEPU580_HEVC_PPS_CU_QP_DLT_EN BIT(13)
+#define VEPU580_HEVC_PPS_CHRM_QP_OFST_PRSN BIT(14)
+#define VEPU580_HEVC_PPS_LP_FLTR_ACRS_SLI BIT(15)
+#define VEPU580_HEVC_PPS_DBLK_FLTR_OVRD_EN BIT(16)
+#define VEPU580_HEVC_PPS_LST_MDFY_PRSNT BIT(17)
+#define VEPU580_HEVC_PPS_SLI_SEG_HDR_EXTN BIT(18)
+#define VEPU580_HEVC_PPS_CU_QP_DLT_DEPTH GENMASK(20, 19)
+#define VEPU580_HEVC_PPS_LPF_FLTR_ACRS_TIL BIT(21)
+#define VEPU580_HEVC_SYNT_SLI0 0x03bc
+#define VEPU580_HEVC_SLI0_CBC_INIT_FLG BIT(0)
+#define VEPU580_HEVC_SLI0_MVD_L1_ZERO_FLG BIT(1)
+#define VEPU580_HEVC_SLI0_MRG_UP_FLG BIT(2)
+#define VEPU580_HEVC_SLI0_MRG_LFT_FLG BIT(3)
+#define VEPU580_HEVC_SLI0_REF_PIC_LST_MDF_L0 BIT(5)
+#define VEPU580_HEVC_SLI0_NUM_REFIDX_L1_ACT GENMASK(7, 6)
+#define VEPU580_HEVC_SLI0_NUM_REFIDX_L0_ACT GENMASK(9, 8)
+#define VEPU580_HEVC_SLI0_NUM_REFIDX_ACT_OVRD BIT(10)
+#define VEPU580_HEVC_SLI0_SAO_CHRM_FLG BIT(11)
+#define VEPU580_HEVC_SLI0_SAO_LUMA_FLG BIT(12)
+#define VEPU580_HEVC_SLI0_TMPRL_MVP_E BIT(13)
+#define VEPU580_HEVC_SLI0_PIC_OUT_FLG BIT(14)
+#define VEPU580_HEVC_SLI0_SLI_TYPE GENMASK(16, 15)
+#define VEPU580_HEVC_SLI0_SLI_RSRV_FLG GENMASK(23, 17)
+#define VEPU580_HEVC_SLI0_DPDNT_SLI_SEG_FLG BIT(24)
+#define VEPU580_HEVC_SLI0_SLI_PPS_ID GENMASK(30, 25)
+#define VEPU580_HEVC_SLI0_NO_OUT_PRI_PIC BIT(31)
+#define VEPU580_HEVC_SYNT_SLI1 0x03c0
+#define VEPU580_HEVC_SLI1_TC_OFST_DIV2 GENMASK(3, 0)
+#define VEPU580_HEVC_SLI1_BETA_OFST_DIV2 GENMASK(7, 4)
+#define VEPU580_HEVC_SLI1_LP_FLTR_ACRS_SLI BIT(8)
+#define VEPU580_HEVC_SLI1_DBLK_FLTR_DIS BIT(9)
+#define VEPU580_HEVC_SLI1_DBLK_FLTR_OVRD_FLG BIT(10)
+#define VEPU580_HEVC_SLI1_CB_QP_OFST GENMASK(15, 11)
+#define VEPU580_HEVC_SLI1_SLI_QP GENMASK(21, 16)
+#define VEPU580_HEVC_SLI1_MAX_MRG_CND GENMASK(23, 22)
+#define VEPU580_HEVC_SLI1_COL_REF_IDX BIT(25)
+#define VEPU580_HEVC_SLI1_COL_FRM_L0_FLG BIT(26)
+#define VEPU580_HEVC_SLI1_LST_ENTRY_L0 GENMASK(30, 27)
+#define VEPU580_HEVC_SYNT_SLI2 0x03c4
+#define VEPU580_HEVC_SLI2_POC_LSB GENMASK(15, 0)
+#define VEPU580_HEVC_SLI2_HDR_EXT_LEN GENMASK(24, 16)
+#define VEPU580_HEVC_SYNT_REFM0 0x03c8
+#define VEPU580_HEVC_REFM0_ST_REF_PIC_FLG BIT(0)
+#define VEPU580_HEVC_REFM0_POC_LSB_LT0 GENMASK(16, 1)
+#define VEPU580_HEVC_REFM0_LT_IDX_SPS GENMASK(21, 17)
+#define VEPU580_HEVC_REFM0_NUM_LT_PIC GENMASK(23, 22)
+#define VEPU580_HEVC_REFM0_ST_REF_PIC_IDX GENMASK(29, 24)
+#define VEPU580_HEVC_REFM0_NUM_LT_SPS GENMASK(31, 30)
+#define VEPU580_HEVC_SYNT_REFM1 0x03cc
+#define VEPU580_HEVC_REFM1_USED_BY_S0 GENMASK(3, 0)
+#define VEPU580_HEVC_REFM1_NUM_POS_PIC BIT(4)
+#define VEPU580_HEVC_REFM1_NUM_NEG_PICS GENMASK(9, 5)
+#define VEPU580_HEVC_REFM1_DLT_POC_MSB_CYCL0 GENMASK(25, 10)
+#define VEPU580_HEVC_REFM1_DLT_POC_MSB_PRSNT(n) BIT(26 + (n))
+#define VEPU580_HEVC_REFM1_USED_BY_LT(n) BIT(29 + (n))
+/* Two delta_poc_s0_minus1 values per register, s0[0] in the low half. */
+#define VEPU580_HEVC_SYNT_REFM2 0x03d0
+#define VEPU580_HEVC_SYNT_REFM3 0x03d4
+#define VEPU580_HEVC_SYNT_LONG_REFM0 0x03d8
+#define VEPU580_HEVC_LONG_REFM0_POC_LSB_LT1 GENMASK(15, 0)
+#define VEPU580_HEVC_LONG_REFM0_POC_LSB_LT2 GENMASK(31, 16)
+#define VEPU580_HEVC_SYNT_LONG_REFM1 0x03dc
+#define VEPU580_HEVC_LONG_REFM1_MSB_CYCL1 GENMASK(15, 0)
+#define VEPU580_HEVC_LONG_REFM1_MSB_CYCL2 GENMASK(31, 16)
+/* 0x3f0 is sli_cfg for H.264 but tile_cfg for HEVC. */
+#define VEPU580_HEVC_TILE_CFG 0x03f0
+#define VEPU580_HEVC_TILE_CFG_TILE_W_M1 GENMASK(7, 0)
+#define VEPU580_HEVC_TILE_CFG_TILE_H_M1 GENMASK(23, 16)
+#define VEPU580_HEVC_TILE_CFG_TILE_EN BIT(31)
+
#define VEPU580_SLI_CFG 0x03f0
#define VEPU580_SLI_CFG_SLI_CRS_EN BIT(31)
#define VEPU580_TILE_POS 0x03f4
@@ -329,6 +430,24 @@
#define VEPU580_PARAM_START 0x1700
#define VEPU580_PARAM_END 0x19d0
+/* Parameter class, HEVC intra pre-selection (written for HEVC only) */
+#define VEPU580_HEVC_PRE_INTRA_CLA(n, b) (0x1b00 + 8 * (n) + 4 * (b))
+#define VEPU580_HEVC_PRE_INTRA_CLA_NUM 17
+#define VEPU580_HEVC_I16_SOBEL_T 0x1c00
+#define VEPU580_HEVC_I16_SOBEL_A(n) (0x1c04 + 4 * (n))
+#define VEPU580_HEVC_I16_SOBEL_B(n) (0x1c0c + 4 * (n))
+#define VEPU580_HEVC_I16_SOBEL_C(n) (0x1c20 + 4 * (n))
+#define VEPU580_HEVC_I16_SOBEL_D(n) (0x1c28 + 4 * (n))
+#define VEPU580_HEVC_I16_SOBEL_E_LOW(qp) (0x1c3c + 8 * (qp))
+#define VEPU580_HEVC_I32_SOBEL_T(n) (0x1c84 + 4 * (n))
+#define VEPU580_HEVC_I32_SOBEL_A 0x1c90
+#define VEPU580_HEVC_I32_SOBEL_B(n) (0x1c94 + 4 * (n))
+#define VEPU580_HEVC_I32_SOBEL_C 0x1ca0
+#define VEPU580_HEVC_I32_SOBEL_D(n) (0x1ca4 + 4 * (n))
+#define VEPU580_HEVC_I32_SOBEL_E_LOW(qp) (0x1cb0 + 8 * (qp))
+#define VEPU580_HEVC_WGT_START 0x1b00
+#define VEPU580_HEVC_WGT_END 0x1cd8
+
/* RDO (SQI) class, H.264 subset */
#define VEPU580_RDO_SQI_CFG 0x2000
#define VEPU580_RDO_SQI_CFG_ATF_PSKIP_EN BIT(0)
@@ -340,6 +459,36 @@
#define VEPU580_RDO_SKIP_CIME_THD1 0x2120
#define VEPU580_RDO_SKIP_VAR_THD(n) (0x2124 + 4 * (n))
#define VEPU580_RDO_SKIP_ATF_WGT(n) (0x2134 + 4 * (n))
+/*
+ * RDO thresholds and weights per CU size, as ten-word blocks: inter/intra
+ * blocks (CIME thresholds 0-2, four variance threshold pairs, four rows of
+ * three weights) and skip blocks (CIME thresholds 0-3, four variance
+ * threshold pairs, weight 00 and rows 1-4).
+ */
+#define VEPU580_HEVC_RDO_B64_INTER 0x2004
+#define VEPU580_HEVC_RDO_B64_SKIP 0x202c
+#define VEPU580_HEVC_RDO_B32_INTRA 0x2054
+#define VEPU580_HEVC_RDO_B32_INTER 0x207c
+#define VEPU580_HEVC_RDO_B32_SKIP 0x20a4
+#define VEPU580_HEVC_RDO_B16_INTRA 0x20cc
+#define VEPU580_HEVC_RDO_B16_INTER 0x20f4
+#define VEPU580_HEVC_RDO_B16_SKIP 0x211c
+#define VEPU580_HEVC_RDO_B8_INTRA 0x2144
+#define VEPU580_HEVC_RDO_B8_INTER 0x216c
+#define VEPU580_HEVC_RDO_B8_SKIP 0x2194
+#define VEPU580_HEVC_RDO_SQI_CFG_SEGMENT_EN BIT(0)
+#define VEPU580_HEVC_RDO_SQI_CFG_SMEAR_EN BIT(1)
+#define VEPU580_HEVC_RDO_SEGMENT_B64_THD0 0x21bc
+#define VEPU580_HEVC_RDO_SEGMENT_B64_THD1 0x21c0
+#define VEPU580_HEVC_RDO_SEGMENT_B32_THD0 0x21c4
+#define VEPU580_HEVC_RDO_SEGMENT_B32_THD1 0x21c8
+#define VEPU580_HEVC_RDO_SEGMENT_MULTI 0x21cc
+#define VEPU580_HEVC_RDO_B16_SMEAR_THD0 0x21d0
+#define VEPU580_HEVC_RDO_B16_SMEAR_THD1 0x21d4
+#define VEPU580_HEVC_PREINTRA_B32_CST_VAR_THD 0x21d8
+#define VEPU580_HEVC_PREINTRA_B32_CST_WGT 0x21dc
+#define VEPU580_HEVC_PREINTRA_B16_CST_VAR_THD 0x21e0
+#define VEPU580_HEVC_PREINTRA_B16_CST_WGT 0x21e4
#define VEPU580_RDO_START 0x2000
#define VEPU580_RDO_END 0x21e8
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
index 3a32caad2216..ab23845a288d 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
@@ -4,8 +4,8 @@
*
* The register meanings and the default tuning values below were derived
* from the Rockchip MPP user space library (hal_h264e_vepu580.c,
- * vepu5xx_common.c, hal_h264e_vepu580_tune.c), which is the only
- * documentation of this block. The values are expressed as named fields
+ * vepu5xx_common.c, hal_h264e_vepu580_tune.c, hal_h265e_vepu580.c), which
+ * is the only documentation of this block. The values are expressed as named fields
* or as formulas instead of register dumps.
*/
@@ -456,6 +456,345 @@ void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
VEPU580_SCL_H264_WORDS * sizeof(u32));
}
+/*
+ * HEVC motion estimation: the same search window as H.264 (+-44 x +-28
+ * pixels), with the CIME cache sized in 64x64 CTU columns and 4-line row
+ * groups.
+ */
+void vepu580_setup_hevc_me(struct rkvenc_core *core, unsigned int width,
+ unsigned int height, bool colmv_stor, bool colmv_load)
+{
+ const unsigned int cache_max = 2464;
+ unsigned int width_mbs = DIV_ROUND_UP(width, 16);
+ unsigned int height_mbs = DIV_ROUND_UP(height, 16);
+ unsigned int cols = DIV_ROUND_UP(width, 64);
+ unsigned int blk_w = 44, blk_h = 28, hal;
+ unsigned int srch_w, srch_h, rx, ry, cur_w, rows, rows_max;
+ unsigned int rama_h, ramb_h, l2_tag;
+
+ if (blk_w / 4 * 2 > (width_mbs * 2 + 1) / 2)
+ blk_w = (width_mbs * 2 + 1) / 2 / 2 * 4;
+
+ hal = ALIGN(height_mbs * 16, 64);
+ if (blk_h / 4 > hal / 128 * 4)
+ blk_h = hal / 128 * 16;
+
+ rkvenc_set(core, VEPU580_ME_RNGE,
+ FIELD_PREP(VEPU580_ME_RNGE_CME_SRCH_H, blk_w / 4) |
+ FIELD_PREP(VEPU580_ME_RNGE_CME_SRCH_V, blk_h / 4) |
+ FIELD_PREP(VEPU580_ME_RNGE_RME_SRCH_H, 7) |
+ FIELD_PREP(VEPU580_ME_RNGE_RME_SRCH_V, 5) |
+ FIELD_PREP(VEPU580_ME_RNGE_DLT_FRM_NUM, 1));
+
+ rkvenc_set(core, VEPU580_ME_CFG,
+ FIELD_PREP(VEPU580_ME_CFG_PMV_MDST_H, 5) |
+ FIELD_PREP(VEPU580_ME_CFG_PMV_MDST_V, 5) |
+ FIELD_PREP(VEPU580_ME_CFG_PMV_NUM, 2) |
+ (colmv_stor ? VEPU580_ME_CFG_COLMV_STOR : 0) |
+ (colmv_load ? VEPU580_ME_CFG_COLMV_LOAD : 0));
+
+ /* Search window in pixels, with a zero global motion vector. */
+ srch_w = blk_w;
+ srch_h = blk_h;
+ rx = (16 - srch_w % 16) % 16;
+ ry = (4 - srch_h % 4) % 4;
+ cur_w = min(DIV_ROUND_UP(rx + 2 * srch_w, 16) + 1, cols);
+ rows = DIV_ROUND_UP(ry + 2 * srch_h, 4) + 4;
+ rows_max = ALIGN(rows, 4);
+
+ rama_h = 4;
+ ramb_h = rows;
+ while (rama_h < rows_max &&
+ cache_max > (rama_h - 4) * cols * 4 + ramb_h * 4 * cur_w) {
+ rama_h += 4;
+ ramb_h = max(ramb_h, 8U) - 4;
+ }
+
+ if (rows == 4) {
+ rama_h += 4;
+ ramb_h = 0;
+ }
+
+ if (cache_max < (rama_h - 4) * cols * 4 + ramb_h * 4 * cur_w)
+ rama_h -= 4;
+
+ if (cols <= 8)
+ l2_tag = 0;
+ else if (cols <= 16)
+ l2_tag = 1;
+ else if (cols <= 32)
+ l2_tag = 2;
+ else
+ l2_tag = 3;
+
+ rkvenc_set(core, VEPU580_ME_CACH,
+ FIELD_PREP(VEPU580_ME_CACH_CME_RAMA_MAX,
+ (rama_h - 4) / 4 * cols + cur_w) |
+ FIELD_PREP(VEPU580_ME_CACH_CME_RAMA_H, rama_h) |
+ FIELD_PREP(VEPU580_ME_CACH_CACH_L2_TAG, l2_tag) |
+ FIELD_PREP(VEPU580_ME_CACH_CME_LINEBUF_W, cols));
+}
+
+/*
+ * Scale a lambda table entry: base[] holds three (or six) consecutive
+ * entries starting at QP @first, and the table doubles every @period QPs.
+ */
+static u32 vepu580_lambda(const u32 *base, unsigned int period,
+ unsigned int first, unsigned int qp)
+{
+ int shift = (int)(qp / period) - (int)(first / period);
+ u32 v = base[qp % period];
+
+ if (shift >= 0)
+ return v << shift;
+
+ return (v + BIT(-shift - 1)) >> -shift;
+}
+
+/* HEVC RDO lambdas (mode decision, intra and inter) and SATD weights. */
+static const u32 vepu580_hevc_lambda_intra[3] = { 0x48f5c3, 0x5bec81, 0x73d119 };
+static const u32 vepu580_hevc_lambda_inter[3] = { 112, 137, 176 };
+static const u32 vepu580_hevc_lambda_satd[6] = {
+ 0xc147, 0xd8f2, 0xf383, 0x11155, 0x132ce, 0x15861,
+};
+
+/* AQ QP steps (the same for I and P pictures) */
+static const s8 vepu580_hevc_aq_step[16] = {
+ -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 5, 7, 8, 9,
+};
+
+/* Intra pre-selection: candidate directions m0..m9 per edge class */
+static const u8 vepu580_hevc_pre_intra_cla[9][10] = {
+ { 10, 11, 12, 13, 14, 9, 15, 8, 16, 7 },
+ { 10, 9, 8, 7, 6, 11, 5, 12, 4, 13 },
+ { 18, 17, 16, 15, 14, 19, 13, 20, 12, 21 },
+ { 18, 19, 20, 21, 22, 17, 23, 16, 24, 15 },
+ { 25, 26, 24, 23, 22, 27, 21, 28, 20, 29 },
+ { 27, 26, 28, 29, 30, 25, 31, 24, 32, 23 },
+ { 34, 33, 32, 31, 30, 2, 29, 3, 28, 4 },
+ { 34, 2, 3, 4, 5, 33, 6, 32, 7, 31 },
+ { 10, 26, 18, 34, 6, 14, 22, 30, 2, 24 },
+};
+
+static u32 pack5x6(const u8 *v)
+{
+ return (v[0] & 0x3f) | (v[1] & 0x3f) << 6 | (v[2] & 0x3f) << 12 |
+ (v[3] & 0x3f) << 18 | (v[4] & 0x3f) << 24;
+}
+
+static u32 pack2x15(u32 lo, u32 hi)
+{
+ return (lo & 0x7fff) | (hi & 0x7fff) << 16;
+}
+
+/* RDO weights of a CU size: four rows of three */
+struct vepu580_hevc_atf {
+ u16 cime[4];
+ u8 wgt00;
+ u8 wgt[4][3];
+};
+
+static const struct vepu580_hevc_atf vepu580_hevc_atf_inter = {
+ .cime = { 32, 144, 300 },
+ .wgt = { { 16, 16, 16 }, { 16, 16, 16 }, { 16, 16, 16 }, { 16, 16, 16 } },
+};
+
+static const struct vepu580_hevc_atf vepu580_hevc_atf_intra = {
+ .cime = { 24, 48, 64 },
+ .wgt = { { 26, 25, 25 }, { 25, 24, 23 }, { 21, 19, 18 }, { 16, 16, 16 } },
+};
+
+static const struct vepu580_hevc_atf vepu580_hevc_atf_skip64 = {
+ .cime = { 24, 4, 6, 8 },
+ .wgt00 = 16,
+ .wgt = { { 10, 10, 10 }, { 14, 14, 15 }, { 15, 15, 16 }, { 16, 16, 16 } },
+};
+
+static const struct vepu580_hevc_atf vepu580_hevc_atf_skip32 = {
+ .cime = { 24, 4, 6, 8 },
+ .wgt00 = 18,
+ .wgt = { { 11, 11, 11 }, { 13, 13, 13 }, { 15, 15, 15 }, { 16, 16, 16 } },
+};
+
+static const struct vepu580_hevc_atf vepu580_hevc_atf_skip16 = {
+ .cime = { 24, 24, 48, 96 },
+ .wgt00 = 16,
+ .wgt = { { 16, 16, 16 }, { 16, 16, 16 }, { 16, 16, 16 }, { 16, 16, 16 } },
+};
+
+static void vepu580_hevc_set_atf(struct rkvenc_core *core, u32 base,
+ const struct vepu580_hevc_atf *atf, bool skip)
+{
+ unsigned int i;
+
+ rkvenc_set(core, base, pack2x12(atf->cime[0], atf->cime[1]));
+ rkvenc_set(core, base + 4, skip ? pack2x12(atf->cime[2], atf->cime[3]) :
+ atf->cime[2]);
+ /* Variance thresholds */
+ for (i = 0; i < 4; i++)
+ rkvenc_set(core, base + 8 + 4 * i, pack2x12(31, 400));
+
+ for (i = 0; i < 4; i++) {
+ const u8 *w = atf->wgt[i];
+
+ if (skip && !i)
+ rkvenc_set(core, base + 24,
+ pack4(atf->wgt00, w[0], w[1], w[2]));
+ else
+ rkvenc_set(core, base + 24 + 4 * i,
+ pack4(w[0], w[1], w[2], 0));
+ }
+}
+
+void vepu580_setup_hevc_tuning(struct rkvenc_core *core, bool intra)
+{
+ static const u8 sobel_a16[5] = { 32, 32, 32, 32, 32 };
+ static const u8 sobel_c16[5] = { 13, 13, 13, 13, 13 };
+ static const u8 sobel_a32[5] = { 18, 18, 18, 18, 18 };
+ static const u8 sobel_c32[5] = { 16, 16, 16, 16, 16 };
+ unsigned int i;
+
+ vepu580_setup_klut(core, intra);
+
+ /* Texture statistics and adaptive QP */
+ rkvenc_set(core, VEPU580_MADI_CFG, FIELD_PREP(VEPU580_MADI_CFG_MADI_THD, 25));
+ rkvenc_set(core, VEPU580_MD_SAD_THD, pack4(20, 30, 40, 0));
+ rkvenc_set(core, VEPU580_MADI_THD, pack4(25, 35, 45, 0));
+ for (i = 0; i < 4; i++) {
+ const s8 *st = &vepu580_hevc_aq_step[4 * i];
+
+ rkvenc_set(core, VEPU580_AQ_TTHD(i),
+ pack4(vepu580_aq_tthd[4 * i], vepu580_aq_tthd[4 * i + 1],
+ vepu580_aq_tthd[4 * i + 2], vepu580_aq_tthd[4 * i + 3]));
+ rkvenc_set(core, VEPU580_AQ_STEP(i),
+ pack4(st[0] & 0x3f, st[1] & 0x3f, st[2] & 0x3f,
+ st[3] & 0x3f));
+ }
+
+ /* Intra cost thresholds (2, 6, 16, 36) and weights of 32x32 and 16x16 */
+ rkvenc_set(core, VEPU580_LVL32_INTRA_CST_THD0, pack2x12(2, 6));
+ rkvenc_set(core, VEPU580_LVL32_INTRA_CST_THD1, pack2x12(16, 36));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_CST_THD0, pack2x12(2, 6));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_CST_THD1, pack2x12(16, 36));
+ rkvenc_set(core, VEPU580_LVL32_INTRA_CST_WGT0, pack4(23, 22, 21, 20));
+ rkvenc_set(core, VEPU580_LVL32_INTRA_CST_WGT1, pack4(22, 24, 26, 0));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_CST_WGT0, pack4(17, 17, 17, 18));
+ rkvenc_set(core, VEPU580_LVL16_INTRA_CST_WGT1, pack4(17, 18, 18, 0));
+
+ /* Quantisation dead zone: 171/1024 (I), 85/1024 (P) */
+ rkvenc_set(core, VEPU580_RDO_QUANT,
+ FIELD_PREP(VEPU580_RDO_QUANT_F_BIAS_I, 171) |
+ FIELD_PREP(VEPU580_RDO_QUANT_F_BIAS_P, 85));
+
+ /* Motion search quality / cost multipliers */
+ rkvenc_set(core, VEPU580_CIME_SQI_CFG,
+ VEPU580_CIME_SQI_CFG_PMV_SET_ZERO |
+ FIELD_PREP(VEPU580_CIME_SQI_CFG_PMV_NUM, 3));
+ rkvenc_set(core, VEPU580_CIME_SQI_THD,
+ FIELD_PREP(VEPU580_CIME_SQI_THD_MVD_TH0, 32) |
+ FIELD_PREP(VEPU580_CIME_SQI_THD_MVD_TH1, 80) |
+ FIELD_PREP(VEPU580_CIME_SQI_THD_MVD_TH2, 128));
+ rkvenc_set(core, VEPU580_CIME_SQI_MULTI0,
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_LO, 16) |
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_HI, 32));
+ rkvenc_set(core, VEPU580_CIME_SQI_MULTI1,
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_LO, 96) |
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_HI, 96));
+ rkvenc_set(core, VEPU580_RIME_SQI_THD,
+ FIELD_PREP(VEPU580_RIME_SQI_THD_CIME_SAD_TH0, 48) |
+ FIELD_PREP(VEPU580_RIME_SQI_THD_MVD_TH0, 3) |
+ FIELD_PREP(VEPU580_RIME_SQI_THD_MVD_TH1, 8));
+ rkvenc_set(core, VEPU580_RIME_SQI_MULTI,
+ FIELD_PREP(VEPU580_RIME_SQI_MULTI0, 16) |
+ FIELD_PREP(VEPU580_RIME_SQI_MULTI1, 16) |
+ FIELD_PREP(VEPU580_RIME_SQI_MULTI2, 128));
+ rkvenc_set(core, VEPU580_FME_SQI_THD0,
+ FIELD_PREP(VEPU580_FME_SQI_THD0_PU16_TH, 16) |
+ FIELD_PREP(VEPU580_FME_SQI_THD0_PU32_TH, 16));
+ rkvenc_set(core, VEPU580_FME_SQI_THD1,
+ FIELD_PREP(VEPU580_FME_SQI_THD1_PU64_TH, 16) |
+ FIELD_PREP(VEPU580_FME_SQI_THD1_MOVE_LAMBDA, 1));
+
+ /*
+ * Lambdas: SATD weights grow by 2^(1/6) per QP, mode decision lambdas
+ * by 2^(1/3).
+ */
+ for (i = 0; i < 52; i++) {
+ rkvenc_set(core, VEPU580_IPRD_WGT_QP(i),
+ vepu580_lambda(vepu580_hevc_lambda_satd, 6, 42, i));
+ rkvenc_set(core, VEPU580_RDO_WGTA_QP(i),
+ intra ? vepu580_lambda(vepu580_hevc_lambda_intra, 3, 48, i) :
+ vepu580_lambda(vepu580_hevc_lambda_inter, 3, 0, i));
+ }
+
+ /* Intra pre-selection classes and Sobel edge weights */
+ for (i = 0; i < ARRAY_SIZE(vepu580_hevc_pre_intra_cla); i++) {
+ rkvenc_set(core, VEPU580_HEVC_PRE_INTRA_CLA(i, 0),
+ pack5x6(&vepu580_hevc_pre_intra_cla[i][0]));
+ rkvenc_set(core, VEPU580_HEVC_PRE_INTRA_CLA(i, 1),
+ pack5x6(&vepu580_hevc_pre_intra_cla[i][5]));
+ }
+
+ rkvenc_set(core, VEPU580_HEVC_I16_SOBEL_T, pack2x12(64, 200));
+ rkvenc_set(core, VEPU580_HEVC_I16_SOBEL_A(0), pack5x6(sobel_a16));
+ rkvenc_set(core, VEPU580_HEVC_I16_SOBEL_A(1), pack5x6(sobel_a16) & 0xffffff);
+ rkvenc_set(core, VEPU580_HEVC_I16_SOBEL_C(0), pack5x6(sobel_c16));
+ rkvenc_set(core, VEPU580_HEVC_I16_SOBEL_C(1), pack5x6(sobel_c16) & 0xffffff);
+ for (i = 0; i < 4; i++)
+ rkvenc_set(core, VEPU580_HEVC_I16_SOBEL_D(i), pack2x15(23750, 23750));
+ rkvenc_set(core, VEPU580_HEVC_I16_SOBEL_D(4), pack2x15(23750, 0));
+ for (i = 0; i < 9; i++)
+ rkvenc_set(core, VEPU580_HEVC_I16_SOBEL_E_LOW(i), 20000);
+
+ rkvenc_set(core, VEPU580_HEVC_I32_SOBEL_T(0), pack2x12(64, 400));
+ rkvenc_set(core, VEPU580_HEVC_I32_SOBEL_T(1), 8);
+ rkvenc_set(core, VEPU580_HEVC_I32_SOBEL_T(2), pack2x12(100, 100));
+ rkvenc_set(core, VEPU580_HEVC_I32_SOBEL_A, pack5x6(sobel_a32));
+ rkvenc_set(core, VEPU580_HEVC_I32_SOBEL_C, pack5x6(sobel_c32));
+ for (i = 0; i < 5; i++)
+ rkvenc_set(core, VEPU580_HEVC_I32_SOBEL_E_LOW(i), 20000);
+
+ /* RDO thresholds and weights per CU size */
+ rkvenc_set(core, VEPU580_RDO_SQI_CFG,
+ VEPU580_HEVC_RDO_SQI_CFG_SEGMENT_EN |
+ VEPU580_HEVC_RDO_SQI_CFG_SMEAR_EN);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B64_INTER, &vepu580_hevc_atf_inter, false);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B64_SKIP, &vepu580_hevc_atf_skip64, true);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B32_INTRA, &vepu580_hevc_atf_intra, false);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B32_INTER, &vepu580_hevc_atf_inter, false);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B32_SKIP, &vepu580_hevc_atf_skip32, true);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B16_INTRA, &vepu580_hevc_atf_intra, false);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B16_INTER, &vepu580_hevc_atf_inter, false);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B16_SKIP, &vepu580_hevc_atf_skip16, true);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B8_INTRA, &vepu580_hevc_atf_intra, false);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B8_INTER, &vepu580_hevc_atf_inter, false);
+ vepu580_hevc_set_atf(core, VEPU580_HEVC_RDO_B8_SKIP, &vepu580_hevc_atf_skip16, true);
+
+ /* Segment (th3..th6 are 4-bit fields for CU64, 2-bit for CU32) */
+ rkvenc_set(core, VEPU580_HEVC_RDO_SEGMENT_B64_THD0, pack2x12(160, 96));
+ rkvenc_set(core, VEPU580_HEVC_RDO_SEGMENT_B64_THD1,
+ 30 | 0 << 16 | 1 << 20 | 4 << 24 | 11 << 28);
+ rkvenc_set(core, VEPU580_HEVC_RDO_SEGMENT_B32_THD0, pack2x12(160, 96));
+ rkvenc_set(core, VEPU580_HEVC_RDO_SEGMENT_B32_THD1,
+ 30 | 0 << 16 | 1 << 20 | 2 << 24 | 3 << 28);
+ rkvenc_set(core, VEPU580_HEVC_RDO_SEGMENT_MULTI, pack4(22, 22, 6, 0));
+ rkvenc_set(core, VEPU580_HEVC_RDO_B16_SMEAR_THD0, pack2x12(64, 32));
+ rkvenc_set(core, VEPU580_HEVC_RDO_B16_SMEAR_THD1, pack2x12(36, 64));
+
+ /* Pre-intra: variance thresholds 9/4, mode threshold 5, weights 31/25 */
+ rkvenc_set(core, VEPU580_HEVC_PREINTRA_B32_CST_VAR_THD,
+ pack2x12(9, 4) | 5 << 29);
+ rkvenc_set(core, VEPU580_HEVC_PREINTRA_B32_CST_WGT, 31 | 25 << 16);
+ rkvenc_set(core, VEPU580_HEVC_PREINTRA_B16_CST_VAR_THD,
+ pack2x12(9, 4) | 5 << 29);
+ rkvenc_set(core, VEPU580_HEVC_PREINTRA_B16_CST_WGT, 31 | 25 << 16);
+
+ /*
+ * The scaling list block is left at zero: with scl_lst_sel = 0 the
+ * hardware quantises with flat matrices.
+ */
+}
+
/*
* Frame-level QP with the hardware row rate control: each block row
* (macroblock row for H.264, CTU row for HEVC) gets target_bits /
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
index 623c50db25a8..26bee08d4c2d 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
@@ -43,6 +43,7 @@ static const struct rkvenc_raw_fmt rkvenc_raw_fmts[] = {
static const struct rkvenc_coded_fmt_desc *rkvenc_coded_fmts[] = {
&rkvenc_h264_fmt_desc,
+ &rkvenc_hevc_fmt_desc,
};
static const struct rkvenc_coded_fmt_desc *rkvenc_find_coded_fmt(u32 fourcc)
@@ -655,6 +656,10 @@ static void rkvenc_hw_start(struct rkvenc_core *core)
rkvenc_write_range(core, VEPU580_FRAME_START, VEPU580_FRAME_END);
rkvenc_write_range(core, VEPU580_KLUT_START, VEPU580_KLUT_END);
rkvenc_write_range(core, VEPU580_PARAM_START, VEPU580_PARAM_END);
+ if (FIELD_GET(VEPU580_ENC_PIC_ENC_STND, core->shadow[VEPU580_ENC_PIC / 4]) ==
+ VEPU580_ENC_STND_HEVC)
+ rkvenc_write_range(core, VEPU580_HEVC_WGT_START,
+ VEPU580_HEVC_WGT_END);
rkvenc_write_range(core, VEPU580_RDO_START, VEPU580_RDO_END);
rkvenc_write_range(core, VEPU580_SCL_START, VEPU580_SCL_END);
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
index 5bc365101600..3515ffc2b490 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
@@ -15,6 +15,7 @@
#include <media/v4l2-ctrls.h>
#include <media/v4l2-device.h>
#include <media/v4l2-h264-enc.h>
+#include <media/v4l2-hevc-enc.h>
#include <media/v4l2-ioctl.h>
#include <media/v4l2-mem2mem.h>
#include <media/videobuf2-core.h>
@@ -45,12 +46,15 @@ struct rkvenc_aux_buf {
* @thumb: 1/4 x 1/4 down-scaled luma used by the coarse motion search
* @colmv: collocated motion vectors (HEVC TMVP only)
* @fbc_hdr_size: offset of the FBC body inside @pixel
+ * @colmv_valid: the hardware stored the motion vectors of the picture in
+ * @colmv (HEVC, inter pictures only)
*/
struct rkvenc_rec_buf {
struct rkvenc_aux_buf pixel;
struct rkvenc_aux_buf thumb;
struct rkvenc_aux_buf colmv;
size_t fbc_hdr_size;
+ bool colmv_valid;
};
enum rkvenc_pnum {
@@ -203,6 +207,14 @@ struct rkvenc_h264_ctx {
bool hw_rc;
};
+/**
+ * struct rkvenc_hevc_ctx - HEVC per-context state
+ */
+struct rkvenc_hevc_ctx {
+ struct v4l2_hevc_enc enc;
+ bool hw_rc;
+};
+
struct rkvenc_ctx {
struct v4l2_fh fh;
struct rkvenc_dev *dev;
@@ -244,6 +256,9 @@ void vepu580_setup_me(struct rkvenc_core *core, unsigned int width_mbs,
bool intra);
void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
struct rkvenc_core *core, bool intra);
+void vepu580_setup_hevc_me(struct rkvenc_core *core, unsigned int width,
+ unsigned int height, bool colmv_stor, bool colmv_load);
+void vepu580_setup_hevc_tuning(struct rkvenc_core *core, bool intra);
/**
* struct vepu580_row_rc - codec-specific row rate control parameters
@@ -271,5 +286,6 @@ dma_addr_t rkvenc_dst_addr(struct rkvenc_run *run);
size_t rkvenc_dst_size(struct rkvenc_run *run);
extern const struct rkvenc_coded_fmt_desc rkvenc_h264_fmt_desc;
+extern const struct rkvenc_coded_fmt_desc rkvenc_hevc_fmt_desc;
#endif /* RKVENC_H_ */
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 05/12] media: rockchip: rkvenc: Don't enable adaptive QP with the row rate control
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (3 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 04/12] media: rockchip: rkvenc: Add HEVC encoding Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 06/12] media: rockchip: rkvenc: Use MPP's moving-scene H.264 RDO tuning Arun Iyengar via B4 Relay
` (6 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The row rate control setup enables the texture adaptive QP (aq_en) with
MPP's MADI threshold and QP step tables, as MPP does. On the board it
costs PSNR at the same bitrate in every case measured: 0.7-1.3 dB for
H.264 (where SSIM drops too) and 0.3-0.7 dB for HEVC (1080p60 clips, 2 to
16 Mb/s, CBR). Leave it off: only the row size deviation moves the QP
within a picture, and the aq_mode field of the row rate control
parameters goes away.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c | 3 +--
drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c | 9 ++++++---
drivers/media/platform/rockchip/rkvenc/rkvenc.h | 2 --
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
index 5bc6960a521a..a3d35a7fdc3f 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
@@ -40,12 +40,11 @@ static const struct v4l2_hevc_enc_rps_limits rkvenc_hevc_rps_limits = {
/*
* Row rate control: MPP's HEVC deviation thresholds are {2n, n, p, 2p}
- * with the adjustments {-2, -1, 0, 1, 2} and AQ mode 1.
+ * with the adjustments {-2, -1, 0, 1, 2}.
*/
static const struct vepu580_row_rc rkvenc_hevc_row_rc = {
.qp_adj = { -2, -1, 0, 1, 2, 0, 0, 0, 0 },
.dthd_outer = 2,
- .aq_mode = true,
};
static int rkvenc_hevc_state_constrain(struct v4l2_hevc_enc *enc,
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
index ab23845a288d..86703dc659f6 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
@@ -805,7 +805,6 @@ void vepu580_setup_hevc_tuning(struct rkvenc_core *core, bool intra)
const struct vepu580_row_rc vepu580_h264_row_rc = {
.qp_adj = { -2, -1, 0, 1, 2, 0, 0, 0, 1 },
.dthd_outer = 4,
- .aq_mode = false,
};
void vepu580_setup_row_rc(struct rkvenc_core *core,
@@ -825,9 +824,13 @@ void vepu580_setup_row_rc(struct rkvenc_core *core,
thd = 5 * row_bits / 16;
rkvenc_or(core, VEPU580_ENC_PIC, FIELD_PREP(VEPU580_ENC_PIC_PIC_QP, qp));
+ /*
+ * MPP also enables the texture adaptive QP here (aq_en). With its
+ * tables that costs 0.3-1.3 dB of PSNR at the same bitrate (and SSIM
+ * for H.264), so the QP only follows the row size deviation.
+ */
rkvenc_set(core, VEPU580_RC_CFG,
- VEPU580_RC_CFG_RC_EN | VEPU580_RC_CFG_AQ_EN |
- (cfg->aq_mode ? VEPU580_RC_CFG_AQ_MODE : 0) |
+ VEPU580_RC_CFG_RC_EN |
FIELD_PREP(VEPU580_RC_CFG_RC_CTU_NUM, blocks_per_row));
rkvenc_set(core, VEPU580_RC_QP,
FIELD_PREP(VEPU580_RC_QP_RANGE, qp_range) |
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
index 3515ffc2b490..c2dd0bb91c51 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
@@ -264,12 +264,10 @@ void vepu580_setup_hevc_tuning(struct rkvenc_core *core, bool intra);
* struct vepu580_row_rc - codec-specific row rate control parameters
* @qp_adj: QP adjustment for each of the nine deviation buckets
* @dthd_outer: multiple of the base threshold for the outer buckets
- * @aq_mode: value of rc_cfg.aq_mode
*/
struct vepu580_row_rc {
s8 qp_adj[9];
unsigned int dthd_outer;
- bool aq_mode;
};
extern const struct vepu580_row_rc vepu580_h264_row_rc;
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 06/12] media: rockchip: rkvenc: Use MPP's moving-scene H.264 RDO tuning
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (4 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 05/12] media: rockchip: rkvenc: Don't enable adaptive QP with the row rate control Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 07/12] media: rockchip: rkvenc: Encode pictures wider than 4096 pixels Arun Iyengar via B4 Relay
` (5 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The H.264 mode decision and motion search registers had the values of
MPP's still-scene tuning set. MPP keeps two sets and switches per picture
from a motion detector (hal_h264e_vepu580_tune.c): the still-scene one
biases the decision towards intra and skip blocks and toward the zero
motion vector against flicker in surveillance video; the moving-scene
one is neutral: ATF weights 16, ATR weights 20, CIME/RIME cost
multipliers 4, no anti-flicker skip or intra, zero PMV off, no CIME SAD
early exits, move lambda 8, P dead zone 341/1024, ATR QP threshold 51,
intra cost thresholds 0 and the intra chroma weights.
Use the moving-scene set for every picture, and start the RDO lambda
table one entry lower than MPP (index 5, lambda / 2^(1/3)). At a fixed QP
(22-36, 1080p60 clips) this needs 7% (animation) to 11% (natural video)
fewer bits for the same PSNR-Y than the still-scene set with MPP's
lambda.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
.../media/platform/rockchip/rkvenc/rkvenc-h264.c | 3 +-
.../platform/rockchip/rkvenc/rkvenc-vepu580.c | 79 +++++++++++-----------
2 files changed, 41 insertions(+), 41 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
index 227d341d060b..665c54278029 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
@@ -333,8 +333,7 @@ static void rkvenc_h264_setup_syntax(struct rkvenc_core *core,
rkvenc_set(core, VEPU580_H264_SYNT_REFM2, 0);
val = VEPU580_RDO_CFG_INTER_4X4 | VEPU580_RDO_CFG_CHRM_SPCL |
- VEPU580_RDO_CFG_CCWA_E | VEPU580_RDO_CFG_ATR_E |
- VEPU580_RDO_CFG_ATF_INTRA_E;
+ VEPU580_RDO_CFG_CCWA_E | VEPU580_RDO_CFG_ATR_E;
if (sps->profile_idc == RKVENC_H264_PROFILE_BASELINE &&
sps->level_idc <= 30)
val |= VEPU580_RDO_CFG_RECT_SIZE;
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
index 86703dc659f6..f14d5ae0b7f7 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
@@ -336,12 +336,18 @@ void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
const s8 *aq_step = intra ? vepu580_aq_step_i : vepu580_aq_step_p;
unsigned int i;
- /* Intra prediction cost / mode decision */
- rkvenc_set(core, VEPU580_IPRD_CSTS,
- FIELD_PREP(VEPU580_IPRD_CSTS_VTHD_Y, 9) |
- FIELD_PREP(VEPU580_IPRD_CSTS_VTHD_C, 63));
+ /*
+ * MPP has two H.264 tuning sets and picks one per picture from a
+ * motion detector: one for still scenes (surveillance cameras), which
+ * biases the mode decision towards intra and skip blocks against
+ * flicker, and a neutral one for moving content. The neutral one is
+ * used for every picture: at the same QP it needs 7-11% fewer bits
+ * for the same PSNR on moving clips. With it the intra prediction
+ * cost thresholds are 0 and the chroma weights start at entry 0.
+ */
+ rkvenc_set(core, VEPU580_IPRD_CSTS, 0);
- vepu580_setup_klut(core, intra);
+ vepu580_setup_klut(core, true);
rkvenc_set(core, VEPU580_MADI_CFG, FIELD_PREP(VEPU580_MADI_CFG_MADI_THD, 25));
rkvenc_set(core, VEPU580_MD_SAD_THD, pack4(4, 9, 15, 0));
@@ -363,23 +369,23 @@ void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
rkvenc_set(core, VEPU580_LVL16_INTRA_CST_THD1, pack2x12(9, 36));
rkvenc_set(core, VEPU580_LVL8_INTRA_CST_THD0, pack2x12(1, 4));
rkvenc_set(core, VEPU580_LVL8_INTRA_CST_THD1, pack2x12(9, 36));
- rkvenc_set(core, VEPU580_LVL16_INTRA_UL_CST_THD, 0);
+ rkvenc_set(core, VEPU580_LVL16_INTRA_UL_CST_THD, 2501);
rkvenc_set(core, VEPU580_LVL32_INTRA_CST_WGT0, pack4(48, 60, 40, 48));
rkvenc_set(core, VEPU580_LVL32_INTRA_CST_WGT1, pack4(48, 60, 40, 48));
rkvenc_set(core, VEPU580_LVL16_INTRA_CST_WGT0, pack4(48, 60, 40, 48));
rkvenc_set(core, VEPU580_LVL16_INTRA_CST_WGT1, pack4(36, 42, 28, 32));
- /* Quantisation dead zone: 683/1024 (I) or 583/1024, 171/1024 (P) */
+ /* Quantisation dead zone: 683/1024 (I) or 583/1024, 341/1024 (P) */
rkvenc_set(core, VEPU580_RDO_QUANT,
FIELD_PREP(VEPU580_RDO_QUANT_F_BIAS_I, intra ? 683 : 583) |
- FIELD_PREP(VEPU580_RDO_QUANT_F_BIAS_P, 171));
+ FIELD_PREP(VEPU580_RDO_QUANT_F_BIAS_P, 341));
/* Adaptive transform (H.264 only) */
if (intra) {
rkvenc_set(core, VEPU580_ATR_THD0, pack2x12(1, 4));
rkvenc_set(core, VEPU580_ATR_THD1,
FIELD_PREP(VEPU580_ATR_THD1_THD2, 36) |
- FIELD_PREP(VEPU580_ATR_THD1_THDQP, 45));
+ FIELD_PREP(VEPU580_ATR_THD1_THDQP, 51));
rkvenc_set(core, VEPU580_LVL16_ATR_WGT, pack4(16, 16, 16, 0));
rkvenc_set(core, VEPU580_LVL8_ATR_WGT, pack4(22, 21, 20, 0));
rkvenc_set(core, VEPU580_LVL4_ATR_WGT, pack4(20, 18, 16, 0));
@@ -387,19 +393,15 @@ void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
rkvenc_set(core, VEPU580_ATR_THD0, pack2x12(4, 16));
rkvenc_set(core, VEPU580_ATR_THD1,
FIELD_PREP(VEPU580_ATR_THD1_THD2, 81) |
- FIELD_PREP(VEPU580_ATR_THD1_THDQP, 45));
- rkvenc_set(core, VEPU580_LVL16_ATR_WGT, pack4(22, 19, 16, 0));
- rkvenc_set(core, VEPU580_LVL8_ATR_WGT, pack4(22, 19, 18, 0));
- rkvenc_set(core, VEPU580_LVL4_ATR_WGT, pack4(22, 19, 16, 0));
+ FIELD_PREP(VEPU580_ATR_THD1_THDQP, 51));
+ rkvenc_set(core, VEPU580_LVL16_ATR_WGT, pack4(20, 20, 20, 0));
+ rkvenc_set(core, VEPU580_LVL8_ATR_WGT, pack4(20, 20, 20, 0));
+ rkvenc_set(core, VEPU580_LVL4_ATR_WGT, pack4(20, 20, 20, 0));
}
- /*
- * Motion search quality / cost multipliers. Where MPP's tuning differs
- * per scene mode, the values of its default (non-IPC) mode are used.
- */
+ /* Motion search quality / cost multipliers (neutral set) */
rkvenc_set(core, VEPU580_CIME_SQI_CFG,
VEPU580_CIME_SQI_CFG_SAD_BIG_BLOCK |
- VEPU580_CIME_SQI_CFG_PMV_SET_ZERO |
FIELD_PREP(VEPU580_CIME_SQI_CFG_PMV_NUM, 3));
rkvenc_set(core, VEPU580_CIME_SQI_THD,
FIELD_PREP(VEPU580_CIME_SQI_THD_MVD_TH0, 32) |
@@ -407,49 +409,48 @@ void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
FIELD_PREP(VEPU580_CIME_SQI_THD_MVD_TH2, 128));
rkvenc_set(core, VEPU580_CIME_SQI_MULTI0,
FIELD_PREP(VEPU580_CIME_SQI_MULTI_LO, 4) |
- FIELD_PREP(VEPU580_CIME_SQI_MULTI_HI, 7));
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_HI, 4));
rkvenc_set(core, VEPU580_CIME_SQI_MULTI1,
- FIELD_PREP(VEPU580_CIME_SQI_MULTI_LO, 20) |
- FIELD_PREP(VEPU580_CIME_SQI_MULTI_HI, 20));
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_LO, 4) |
+ FIELD_PREP(VEPU580_CIME_SQI_MULTI_HI, 4));
rkvenc_set(core, VEPU580_RIME_SQI_THD,
- FIELD_PREP(VEPU580_RIME_SQI_THD_CIME_SAD_TH0, 50) |
FIELD_PREP(VEPU580_RIME_SQI_THD_MVD_TH0, 3) |
FIELD_PREP(VEPU580_RIME_SQI_THD_MVD_TH1, 8));
rkvenc_set(core, VEPU580_RIME_SQI_MULTI,
FIELD_PREP(VEPU580_RIME_SQI_MULTI0, 4) |
- FIELD_PREP(VEPU580_RIME_SQI_MULTI1, 16) |
- FIELD_PREP(VEPU580_RIME_SQI_MULTI2, 64));
- rkvenc_set(core, VEPU580_FME_SQI_THD0,
- FIELD_PREP(VEPU580_FME_SQI_THD0_PU16_TH, 2));
+ FIELD_PREP(VEPU580_RIME_SQI_MULTI1, 4) |
+ FIELD_PREP(VEPU580_RIME_SQI_MULTI2, 4));
+ rkvenc_set(core, VEPU580_FME_SQI_THD0, 0);
rkvenc_set(core, VEPU580_FME_SQI_THD1,
- FIELD_PREP(VEPU580_FME_SQI_THD1_MOVE_LAMBDA, 1));
+ FIELD_PREP(VEPU580_FME_SQI_THD1_MOVE_LAMBDA, 8));
- /* RDO lambda, intra prediction weights stay zero for H.264 */
+ /*
+ * RDO lambda, intra prediction weights stay zero for H.264. MPP
+ * starts at entry 6 (its lambda_idx); entry 5, one 2^(1/3) step
+ * lower, saves 3-4% of the bits at the same PSNR.
+ */
for (i = 0; i < 52; i++)
- rkvenc_set(core, VEPU580_RDO_WGTA_QP(i), vepu580_h264_lambda[i + 6]);
+ rkvenc_set(core, VEPU580_RDO_WGTA_QP(i), vepu580_h264_lambda[i + 5]);
- /* RDO intra / skip thresholds and weights */
- rkvenc_set(core, VEPU580_RDO_SQI_CFG, VEPU580_RDO_SQI_CFG_ATF_PSKIP_EN);
+ /* RDO intra / skip thresholds and weights; no anti-flicker skips */
+ rkvenc_set(core, VEPU580_RDO_SQI_CFG, 0);
rkvenc_set(core, VEPU580_RDO_INTRA_CIME_THD0, pack2x12(28, 44));
rkvenc_set(core, VEPU580_RDO_INTRA_CIME_THD1, pack2x12(72, 0));
rkvenc_set(core, VEPU580_RDO_INTRA_VAR_THD(0), pack2x12(25, 64));
rkvenc_set(core, VEPU580_RDO_INTRA_VAR_THD(1), pack2x12(25, 64));
rkvenc_set(core, VEPU580_RDO_INTRA_VAR_THD(2), pack2x12(70, 100));
rkvenc_set(core, VEPU580_RDO_INTRA_VAR_THD(3), pack2x12(70, 100));
- rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(0), pack4(22, 21, 20, 0));
- rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(1), pack4(21, 20, 19, 0));
- rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(2), pack4(20, 19, 18, 0));
- rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(3), pack4(16, 16, 16, 0));
+ for (i = 0; i < 4; i++)
+ rkvenc_set(core, VEPU580_RDO_INTRA_ATF_WGT(i), pack4(16, 16, 16, 0));
rkvenc_set(core, VEPU580_RDO_SKIP_CIME_THD0, pack2x12(10, 8));
rkvenc_set(core, VEPU580_RDO_SKIP_CIME_THD1, pack2x12(15, 25));
rkvenc_set(core, VEPU580_RDO_SKIP_VAR_THD(0), pack2x12(25, 40));
rkvenc_set(core, VEPU580_RDO_SKIP_VAR_THD(1), pack2x12(25, 40));
rkvenc_set(core, VEPU580_RDO_SKIP_VAR_THD(2), pack2x12(70, 100));
rkvenc_set(core, VEPU580_RDO_SKIP_VAR_THD(3), pack2x12(70, 100));
- rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(0), pack4(20, 16, 16, 16));
- rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(1), pack4(16, 16, 16, 0));
- rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(2), pack4(16, 16, 16, 0));
- rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(3), pack4(16, 16, 16, 0));
+ rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(0), pack4(16, 16, 16, 16));
+ for (i = 1; i < 4; i++)
+ rkvenc_set(core, VEPU580_RDO_SKIP_ATF_WGT(i), pack4(16, 16, 16, 0));
/* Flat quantisation tables */
memcpy(&core->shadow[VEPU580_SCL_START / 4], rkvenc->h264_scl,
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 07/12] media: rockchip: rkvenc: Encode pictures wider than 4096 pixels
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (5 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 06/12] media: rockchip: rkvenc: Use MPP's moving-scene H.264 RDO tuning Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 08/12] media: rockchip: rkvenc: Encode HEVC wider than 4096 pixels in tile columns Arun Iyengar via B4 Relay
` (4 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The internal line buffers of the VEPU580 cover pictures up to 4096
pixels wide. Wider pictures need an external line buffer of 480 bytes per
64-pixel column, given to the encoder by its bottom address and its
(exclusive) top address in ADR_ELBB and ADR_ELBT. Both registers hold
address bits 31:8.
Allocate that buffer per context at stream start when the encoded width
is above 4096, program it for every picture, and raise the frame size
limits of both codecs to the 16384x8192 the encoder supports.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
.../media/platform/rockchip/rkvenc/rkvenc-h264.c | 6 ++--
.../media/platform/rockchip/rkvenc/rkvenc-hevc.c | 6 ++--
.../media/platform/rockchip/rkvenc/rkvenc-regs.h | 2 ++
.../platform/rockchip/rkvenc/rkvenc-vepu580.c | 38 ++++++++++++++++++++--
drivers/media/platform/rockchip/rkvenc/rkvenc.c | 8 ++++-
drivers/media/platform/rockchip/rkvenc/rkvenc.h | 4 +++
6 files changed, 54 insertions(+), 10 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
index 665c54278029..11051aea43b4 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
@@ -501,11 +501,11 @@ const struct rkvenc_coded_fmt_desc rkvenc_h264_fmt_desc = {
.size_align = 16,
.frmsize = {
.min_width = 64,
- /* The external line buffer for wider pictures is not handled. */
- .max_width = 4096,
+ /* Wider than 4096 with the external line buffer. */
+ .max_width = 16384,
.step_width = 2,
.min_height = 64,
- .max_height = 4096,
+ .max_height = 8192,
.step_height = 2,
},
.ctrls = &rkvenc_h264_ctrls,
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
index a3d35a7fdc3f..638b13d6642a 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
@@ -689,11 +689,11 @@ const struct rkvenc_coded_fmt_desc rkvenc_hevc_fmt_desc = {
.size_align = 8,
.frmsize = {
.min_width = 64,
- /* Wider pictures need tiles, which are not implemented. */
- .max_width = 4096,
+ /* Wider than 4096 with the external line buffer. */
+ .max_width = 16384,
.step_width = 2,
.min_height = 64,
- .max_height = 4096,
+ .max_height = 8192,
.step_height = 2,
},
.ctrls = &rkvenc_hevc_ctrls,
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
index e8e7ce045f90..dd91004e31cf 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
@@ -82,6 +82,8 @@
#define VEPU580_LPFR_ADDR 0x02c4
#define VEPU580_EBUFT_ADDR 0x02d8
#define VEPU580_EBUFB_ADDR 0x02dc
+/* Widest picture the internal line buffers cover. */
+#define VEPU580_LINE_BUF_MAX_WIDTH 4096
/* Frame class: control */
#define VEPU580_ENC_PIC 0x0300
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
index f14d5ae0b7f7..adb1372dff57 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
@@ -179,9 +179,41 @@ void vepu580_setup_common(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
rkvenc_set(core, VEPU580_SLI_SPLT, 0);
rkvenc_set(core, VEPU580_SLI_CFG, VEPU580_SLI_CFG_SLI_CRS_EN);
- /* No external line buffer: widths above 4096 are rejected for now. */
- rkvenc_set(core, VEPU580_EBUFT_ADDR, 0);
- rkvenc_set(core, VEPU580_EBUFB_ADDR, 0);
+ /* The external line buffer: top is the exclusive end, 0 when unused. */
+ if (ctx->line_buf.cpu) {
+ rkvenc_set(core, VEPU580_EBUFT_ADDR,
+ ctx->line_buf.dma + ctx->line_buf.size);
+ rkvenc_set(core, VEPU580_EBUFB_ADDR, ctx->line_buf.dma);
+ } else {
+ rkvenc_set(core, VEPU580_EBUFT_ADDR, 0);
+ rkvenc_set(core, VEPU580_EBUFB_ADDR, 0);
+ }
+}
+
+/*
+ * The internal line buffers cover pictures up to 4096 pixels wide. Wider
+ * pictures need an external one of 480 bytes per 64-pixel column. Its
+ * address registers hold bits 31:8 only, hence the 256-byte rounding (the
+ * DMA allocation itself is page aligned). The content does not outlive a
+ * picture, and a context runs one picture at a time, so one buffer per
+ * context is enough.
+ */
+int vepu580_alloc_line_buf(struct rkvenc_ctx *ctx)
+{
+ unsigned int width = ALIGN(ctx->coded_fmt.fmt.pix_mp.width,
+ ctx->coded_fmt_desc->size_align);
+
+ if (width <= VEPU580_LINE_BUF_MAX_WIDTH)
+ return 0;
+
+ return rkvenc_alloc_aux(ctx->dev, &ctx->line_buf,
+ ALIGN(DIV_ROUND_UP(width, 64) * 480, 256),
+ false);
+}
+
+void vepu580_free_line_buf(struct rkvenc_ctx *ctx)
+{
+ rkvenc_free_aux(ctx->dev, &ctx->line_buf);
}
/*
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
index 26bee08d4c2d..61c745994da9 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
@@ -505,7 +505,12 @@ static int rkvenc_start_streaming(struct vb2_queue *q, unsigned int count)
ctx->src_fmt.fmt.pix_mp.height != ctx->coded_fmt.fmt.pix_mp.height)
ret = -EINVAL;
else
- ret = desc->ops->start ? desc->ops->start(ctx) : 0;
+ ret = vepu580_alloc_line_buf(ctx);
+ if (!ret && desc->ops->start) {
+ ret = desc->ops->start(ctx);
+ if (ret)
+ vepu580_free_line_buf(ctx);
+ }
if (ret) {
rkvenc_queue_cleanup(q, VB2_BUF_STATE_QUEUED);
return ret;
@@ -524,6 +529,7 @@ static void rkvenc_stop_streaming(struct vb2_queue *q)
if (ctx->started) {
if (desc->ops->stop)
desc->ops->stop(ctx);
+ vepu580_free_line_buf(ctx);
ctx->started = false;
}
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
index c2dd0bb91c51..ec6a955921b8 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
@@ -226,6 +226,8 @@ struct rkvenc_ctx {
struct v4l2_ctrl_handler ctrl_hdl;
struct rkvenc_core *core;
struct v4l2_ctrl *avg_qp_ctrl;
+ /* External line buffer, only for pictures wider than 4096 pixels. */
+ struct rkvenc_aux_buf line_buf;
void *priv;
bool started;
};
@@ -249,6 +251,8 @@ static inline void rkvenc_or(struct rkvenc_core *core, u32 reg, u32 val)
int rkvenc_alloc_aux(struct rkvenc_dev *rkvenc, struct rkvenc_aux_buf *buf,
size_t size, bool kernel_mapping);
void rkvenc_free_aux(struct rkvenc_dev *rkvenc, struct rkvenc_aux_buf *buf);
+int vepu580_alloc_line_buf(struct rkvenc_ctx *ctx);
+void vepu580_free_line_buf(struct rkvenc_ctx *ctx);
void vepu580_setup_common(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
struct rkvenc_run *run);
void vepu580_setup_me(struct rkvenc_core *core, unsigned int width_mbs,
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 08/12] media: rockchip: rkvenc: Encode HEVC wider than 4096 pixels in tile columns
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (6 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 07/12] media: rockchip: rkvenc: Encode pictures wider than 4096 pixels Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 09/12] media: rockchip: rkvenc: Limit the QP of the HEVC I pictures to 15..51 Arun Iyengar via B4 Relay
` (3 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The VEPU580 HEVC encoder does not keep the state of the CTU row above in
the external line buffer: with it, HEVC pictures wider than 4096 pixels
come out corrupted all over, with CABAC desynchronisation in the decoder,
while H.264 pictures of the same sizes are fine. Rockchip's MPP never uses
the line buffer for HEVC on this encoder either; it splits wider pictures
into tile columns of at most 4096 pixels and encodes each with its own
hardware pass.
Do the same: above 4096 pixels, signal ceil(width / 4096) uniformly
spaced tile columns, with loop filtering across them, in the PPS, and
encode the picture in one pass per tile column. Each pass gets the size
and position of its tile, the motion search cache for the CTU columns its
search window can reach, the CTUs per row of the row rate control, and
the buffers that carry the loop filter data of the tile edges from one
tile to the next. It continues the bitstream after the previous one as a
slice of its own; the status of the passes is summed for the picture.
Keep the default L2 cache mapping for pictures wider than 4096, as MPP
does (the 4x4096 mapping covers at most 4096 columns), and allocate the
external line buffer for H.264 only.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
.../media/platform/rockchip/rkvenc/rkvenc-h264.c | 1 +
.../media/platform/rockchip/rkvenc/rkvenc-hevc.c | 141 ++++++++++++++++++++-
.../media/platform/rockchip/rkvenc/rkvenc-regs.h | 2 +
.../platform/rockchip/rkvenc/rkvenc-vepu580.c | 100 +++++++++------
drivers/media/platform/rockchip/rkvenc/rkvenc.c | 48 +++++++
drivers/media/platform/rockchip/rkvenc/rkvenc.h | 31 +++++
6 files changed, 281 insertions(+), 42 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
index 11051aea43b4..fcb8c070ab30 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
@@ -499,6 +499,7 @@ static const struct rkvenc_ctrls rkvenc_h264_ctrls = {
const struct rkvenc_coded_fmt_desc rkvenc_h264_fmt_desc = {
.fourcc = V4L2_PIX_FMT_H264_SLICE,
.size_align = 16,
+ .ext_line_buf = true,
.frmsize = {
.min_width = 64,
/* Wider than 4096 with the external line buffer. */
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
index 638b13d6642a..27006e393121 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
@@ -28,6 +28,8 @@
#include "rkvenc.h"
#define RKVENC_HEVC_CTB_SIZE 64
+/* Widest tile column, in CTUs, the internal line buffers cover */
+#define RKVENC_HEVC_TILE_MAX_CTBS (VEPU580_LINE_BUF_MAX_WIDTH / RKVENC_HEVC_CTB_SIZE)
/* max_mrg_cnd is a 2-bit field: MaxNumMergeCand 1..3 */
#define RKVENC_HEVC_MAX_MERGE_CAND 3
@@ -47,6 +49,23 @@ static const struct vepu580_row_rc rkvenc_hevc_row_rc = {
.dthd_outer = 2,
};
+/*
+ * The HEVC encoder does not use the external line buffer: as in MPP,
+ * pictures wider than 4096 pixels are split into uniformly spaced tile
+ * columns of at most 4096 pixels, each encoded by its own hardware pass.
+ */
+static unsigned int rkvenc_hevc_tile_cols(unsigned int width_ctbs)
+{
+ return DIV_ROUND_UP(width_ctbs, RKVENC_HEVC_TILE_MAX_CTBS);
+}
+
+/* First CTU column of tile column @i, the uniform spacing of (6-3). */
+static unsigned int rkvenc_hevc_tile_start(unsigned int width_ctbs,
+ unsigned int cols, unsigned int i)
+{
+ return i * width_ctbs / cols;
+}
+
static int rkvenc_hevc_state_constrain(struct v4l2_hevc_enc *enc,
struct v4l2_hevc_enc_state *state)
{
@@ -119,6 +138,13 @@ static int rkvenc_hevc_state_constrain(struct v4l2_hevc_enc *enc,
V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT);
pps->num_tile_columns_minus1 = 0;
pps->num_tile_rows_minus1 = 0;
+ if (rkvenc_hevc_tile_cols(state->width_ctbs) > 1) {
+ pps->flags |= V4L2_HEVC_PPS_FLAG_TILES_ENABLED |
+ V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING |
+ V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED;
+ pps->num_tile_columns_minus1 =
+ rkvenc_hevc_tile_cols(state->width_ctbs) - 1;
+ }
pps->num_extra_slice_header_bits = 0;
pps->log2_parallel_merge_level_minus2 = 0;
@@ -213,6 +239,27 @@ static const struct v4l2_hevc_enc_ops rkvenc_hevc_enc_ops = {
/* ---- codec ops ---- */
+/*
+ * Each tile column but the last stores the loop filter and frame buffer
+ * compression data of its right edge for the next one: one CTU row more than
+ * the TRM's 2 KiB per CTU row, as in MPP, in 1 KiB aligned parts.
+ */
+static int rkvenc_hevc_tile_buf_alloc(struct rkvenc_ctx *ctx,
+ struct rkvenc_hevc_ctx *hevc)
+{
+ const struct v4l2_pix_format_mplane *pix = &ctx->coded_fmt.fmt.pix_mp;
+ unsigned int width_ctbs = DIV_ROUND_UP(pix->width, RKVENC_HEVC_CTB_SIZE);
+ unsigned int height_ctbs = DIV_ROUND_UP(pix->height, RKVENC_HEVC_CTB_SIZE);
+ unsigned int cols = rkvenc_hevc_tile_cols(width_ctbs);
+
+ if (cols < 2)
+ return 0;
+
+ hevc->tile_buf_size = ALIGN((height_ctbs + 1) * SZ_2K, SZ_1K);
+ return rkvenc_alloc_aux(ctx->dev, &hevc->tile_buf,
+ hevc->tile_buf_size * cols, false);
+}
+
static int rkvenc_hevc_start(struct rkvenc_ctx *ctx)
{
struct rkvenc_hevc_ctx *hevc;
@@ -236,8 +283,15 @@ static int rkvenc_hevc_start(struct rkvenc_ctx *ctx)
enc->flags = V4L2_HEVC_ENC_FLAG_INTER_PRED |
V4L2_HEVC_ENC_FLAG_HW_SLICE_HEADER;
+ ret = rkvenc_hevc_tile_buf_alloc(ctx, hevc);
+ if (ret) {
+ kfree(hevc);
+ return ret;
+ }
+
ret = v4l2_hevc_enc_init(enc);
if (ret) {
+ rkvenc_free_aux(ctx->dev, &hevc->tile_buf);
kfree(hevc);
return ret;
}
@@ -254,6 +308,7 @@ static void rkvenc_hevc_stop(struct rkvenc_ctx *ctx)
return;
v4l2_hevc_enc_exit(&hevc->enc);
+ rkvenc_free_aux(ctx->dev, &hevc->tile_buf);
kfree(hevc);
ctx->priv = NULL;
}
@@ -355,6 +410,8 @@ static void rkvenc_hevc_setup_syntax(struct v4l2_hevc_enc *enc,
val |= VEPU580_HEVC_PPS_LP_FLTR_ACRS_SLI;
if (pps->flags & V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED)
val |= VEPU580_HEVC_PPS_DBLK_FLTR_OVRD_EN;
+ if (pps->flags & V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED)
+ val |= VEPU580_HEVC_PPS_LPF_FLTR_ACRS_TIL;
rkvenc_set(core, VEPU580_HEVC_SYNT_PPS, val);
val = FIELD_PREP(VEPU580_HEVC_SLI0_SLI_TYPE, sh->slice_type) |
@@ -456,10 +513,58 @@ static void rkvenc_hevc_setup_syntax(struct v4l2_hevc_enc *enc,
FIELD_PREP(VEPU580_HEVC_RDO_CFG_CU_INTER_E, 0x6db) |
FIELD_PREP(VEPU580_HEVC_RDO_CFG_CU_INTRA_E, 0xf) |
VEPU580_HEVC_RDO_CFG_CCWA_E);
+}
- /* 0x3f0 is tile_cfg for HEVC: a single tile. */
- rkvenc_set(core, VEPU580_HEVC_TILE_CFG, 0);
- rkvenc_set(core, VEPU580_TILE_POS, 0);
+/*
+ * Program the tile column @tile of the picture: its size and position, the
+ * motion search cache for the columns it can reach, the CTUs per row of the
+ * row rate control and the loop filter buffers of its left (read) and right
+ * (written) edges. The other registers are the same for all the tiles.
+ */
+static void rkvenc_hevc_setup_tile(struct rkvenc_hevc_ctx *hevc,
+ struct rkvenc_core *core, unsigned int tile)
+{
+ const struct v4l2_hevc_enc_state *state = &hevc->enc.state_active;
+ const struct v4l2_ctrl_hevc_pps *pps = &state->pps;
+ unsigned int cols = pps->num_tile_columns_minus1 + 1;
+ unsigned int width = state->sps.pic_width_in_luma_samples;
+ unsigned int x, w;
+ dma_addr_t buf;
+
+ hevc->tile = tile;
+ if (!(pps->flags & V4L2_HEVC_PPS_FLAG_TILES_ENABLED)) {
+ /* 0x3f0 is tile_cfg for HEVC: a single tile. */
+ rkvenc_set(core, VEPU580_HEVC_TILE_CFG, 0);
+ rkvenc_set(core, VEPU580_TILE_POS, 0);
+ rkvenc_set(core, VEPU580_LPFW_ADDR, 0);
+ rkvenc_set(core, VEPU580_LPFR_ADDR, 0);
+ vepu580_setup_hevc_cime(core, width, 0, state->width_ctbs);
+ return;
+ }
+
+ x = rkvenc_hevc_tile_start(state->width_ctbs, cols, tile);
+ w = rkvenc_hevc_tile_start(state->width_ctbs, cols, tile + 1) - x;
+
+ rkvenc_set(core, VEPU580_HEVC_TILE_CFG,
+ FIELD_PREP(VEPU580_HEVC_TILE_CFG_TILE_W_M1, w - 1) |
+ FIELD_PREP(VEPU580_HEVC_TILE_CFG_TILE_H_M1,
+ state->height_ctbs - 1) |
+ VEPU580_HEVC_TILE_CFG_TILE_EN);
+ rkvenc_set(core, VEPU580_TILE_POS,
+ FIELD_PREP(VEPU580_TILE_POS_TILE_X, x) |
+ FIELD_PREP(VEPU580_TILE_POS_TILE_Y, 0));
+
+ vepu580_setup_hevc_cime(core, width, x, w);
+
+ if (hevc->hw_rc)
+ rkvenc_update(core, VEPU580_RC_CFG, VEPU580_RC_CFG_RC_CTU_NUM,
+ FIELD_PREP(VEPU580_RC_CFG_RC_CTU_NUM, w));
+
+ /* The first tile reads nothing, the last one writes nothing. */
+ buf = hevc->tile_buf.dma;
+ rkvenc_set(core, VEPU580_LPFW_ADDR, buf + tile * hevc->tile_buf_size);
+ rkvenc_set(core, VEPU580_LPFR_ADDR,
+ buf + (tile ? tile - 1 : 1) * hevc->tile_buf_size);
}
static int rkvenc_hevc_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
@@ -558,9 +663,36 @@ static int rkvenc_hevc_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
else
vepu580_setup_fixed_qp(core, enc->qp);
+ rkvenc_hevc_setup_tile(hevc, core, 0);
+
return 0;
}
+/*
+ * The next tile column continues the bitstream right after the previous
+ * one, as a slice of its own (the hardware writes its slice header).
+ */
+static int rkvenc_hevc_next_pass(struct rkvenc_ctx *ctx,
+ struct rkvenc_core *core,
+ struct rkvenc_run *run)
+{
+ struct rkvenc_hevc_ctx *hevc = ctx->priv;
+ const struct v4l2_ctrl_hevc_pps *pps = &hevc->enc.state_active.pps;
+ u64 offset = run->hdr_bytes + run->fb.bs_len;
+
+ if (!(pps->flags & V4L2_HEVC_PPS_FLAG_TILES_ENABLED) ||
+ hevc->tile >= pps->num_tile_columns_minus1)
+ return 0;
+
+ if (!run->fb.bs_len || offset + SZ_4K > rkvenc_dst_size(run))
+ return -ENOSPC;
+
+ rkvenc_set(core, VEPU580_ADR_BSBS, rkvenc_dst_addr(run) + offset);
+ rkvenc_hevc_setup_tile(hevc, core, hevc->tile + 1);
+
+ return 1;
+}
+
static void rkvenc_hevc_done(struct rkvenc_ctx *ctx, struct rkvenc_run *run,
const struct rkvenc_feedback *fb,
enum vb2_buffer_state *state)
@@ -628,6 +760,7 @@ static const struct rkvenc_codec_ops rkvenc_hevc_ops = {
.start = rkvenc_hevc_start,
.stop = rkvenc_hevc_stop,
.run = rkvenc_hevc_run,
+ .next_pass = rkvenc_hevc_next_pass,
.done = rkvenc_hevc_done,
};
@@ -689,7 +822,7 @@ const struct rkvenc_coded_fmt_desc rkvenc_hevc_fmt_desc = {
.size_align = 8,
.frmsize = {
.min_width = 64,
- /* Wider than 4096 with the external line buffer. */
+ /* Wider than 4096 in tile columns of up to 4096. */
.max_width = 16384,
.step_width = 2,
.min_height = 64,
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
index dd91004e31cf..e3843aee69cc 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-regs.h
@@ -354,6 +354,8 @@
#define VEPU580_SLI_CFG 0x03f0
#define VEPU580_SLI_CFG_SLI_CRS_EN BIT(31)
#define VEPU580_TILE_POS 0x03f4
+#define VEPU580_TILE_POS_TILE_X GENMASK(7, 0)
+#define VEPU580_TILE_POS_TILE_Y GENMASK(23, 16)
#define VEPU580_FRAME_START 0x0280
#define VEPU580_FRAME_END 0x03f8
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
index adb1372dff57..3b52b56e8dfa 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu580.c
@@ -203,7 +203,8 @@ int vepu580_alloc_line_buf(struct rkvenc_ctx *ctx)
unsigned int width = ALIGN(ctx->coded_fmt.fmt.pix_mp.width,
ctx->coded_fmt_desc->size_align);
- if (width <= VEPU580_LINE_BUF_MAX_WIDTH)
+ if (!ctx->coded_fmt_desc->ext_line_buf ||
+ width <= VEPU580_LINE_BUF_MAX_WIDTH)
return 0;
return rkvenc_alloc_aux(ctx->dev, &ctx->line_buf,
@@ -490,45 +491,29 @@ void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
}
/*
- * HEVC motion estimation: the same search window as H.264 (+-44 x +-28
- * pixels), with the CIME cache sized in 64x64 CTU columns and 4-line row
- * groups.
+ * HEVC coarse integer motion estimation (CIME) cache, sized in 64x64 CTU
+ * columns and 4-line row groups, for the CTU columns the search window can
+ * reach from a tile of @tile_w CTU columns starting at column @tile_x (the
+ * whole picture when it is not split into tiles).
*/
-void vepu580_setup_hevc_me(struct rkvenc_core *core, unsigned int width,
- unsigned int height, bool colmv_stor, bool colmv_load)
+void vepu580_setup_hevc_cime(struct rkvenc_core *core, unsigned int width,
+ unsigned int tile_x, unsigned int tile_w)
{
const unsigned int cache_max = 2464;
- unsigned int width_mbs = DIV_ROUND_UP(width, 16);
- unsigned int height_mbs = DIV_ROUND_UP(height, 16);
- unsigned int cols = DIV_ROUND_UP(width, 64);
- unsigned int blk_w = 44, blk_h = 28, hal;
- unsigned int srch_w, srch_h, rx, ry, cur_w, rows, rows_max;
+ u32 rnge = core->shadow[VEPU580_ME_RNGE / 4];
+ unsigned int srch_w = FIELD_GET(VEPU580_ME_RNGE_CME_SRCH_H, rnge) * 4;
+ unsigned int srch_h = FIELD_GET(VEPU580_ME_RNGE_CME_SRCH_V, rnge) * 4;
+ unsigned int pic_cols = DIV_ROUND_UP(width, 64);
+ unsigned int reach = DIV_ROUND_UP(srch_w, 16);
+ unsigned int first, last, cols;
+ unsigned int rx, ry, cur_w, rows, rows_max;
unsigned int rama_h, ramb_h, l2_tag;
- if (blk_w / 4 * 2 > (width_mbs * 2 + 1) / 2)
- blk_w = (width_mbs * 2 + 1) / 2 / 2 * 4;
-
- hal = ALIGN(height_mbs * 16, 64);
- if (blk_h / 4 > hal / 128 * 4)
- blk_h = hal / 128 * 16;
-
- rkvenc_set(core, VEPU580_ME_RNGE,
- FIELD_PREP(VEPU580_ME_RNGE_CME_SRCH_H, blk_w / 4) |
- FIELD_PREP(VEPU580_ME_RNGE_CME_SRCH_V, blk_h / 4) |
- FIELD_PREP(VEPU580_ME_RNGE_RME_SRCH_H, 7) |
- FIELD_PREP(VEPU580_ME_RNGE_RME_SRCH_V, 5) |
- FIELD_PREP(VEPU580_ME_RNGE_DLT_FRM_NUM, 1));
-
- rkvenc_set(core, VEPU580_ME_CFG,
- FIELD_PREP(VEPU580_ME_CFG_PMV_MDST_H, 5) |
- FIELD_PREP(VEPU580_ME_CFG_PMV_MDST_V, 5) |
- FIELD_PREP(VEPU580_ME_CFG_PMV_NUM, 2) |
- (colmv_stor ? VEPU580_ME_CFG_COLMV_STOR : 0) |
- (colmv_load ? VEPU580_ME_CFG_COLMV_LOAD : 0));
-
/* Search window in pixels, with a zero global motion vector. */
- srch_w = blk_w;
- srch_h = blk_h;
+ first = tile_x > reach ? tile_x - reach : 0;
+ last = min(tile_x + tile_w - 1 + reach, pic_cols - 1);
+ cols = last - first + 1;
+
rx = (16 - srch_w % 16) % 16;
ry = (4 - srch_h % 4) % 4;
cur_w = min(DIV_ROUND_UP(rx + 2 * srch_w, 16) + 1, cols);
@@ -551,14 +536,20 @@ void vepu580_setup_hevc_me(struct rkvenc_core *core, unsigned int width,
if (cache_max < (rama_h - 4) * cols * 4 + ramb_h * 4 * cur_w)
rama_h -= 4;
- if (cols <= 8)
+ /*
+ * The L2 cache maps at most 4096 pixel columns: wider pictures (which
+ * are split into tiles) keep the default mapping, as in MPP.
+ */
+ if (pic_cols <= 8)
l2_tag = 0;
- else if (cols <= 16)
+ else if (pic_cols <= 16)
l2_tag = 1;
- else if (cols <= 32)
+ else if (pic_cols <= 32)
l2_tag = 2;
- else
+ else if (pic_cols <= 64)
l2_tag = 3;
+ else
+ l2_tag = 0;
rkvenc_set(core, VEPU580_ME_CACH,
FIELD_PREP(VEPU580_ME_CACH_CME_RAMA_MAX,
@@ -568,6 +559,39 @@ void vepu580_setup_hevc_me(struct rkvenc_core *core, unsigned int width,
FIELD_PREP(VEPU580_ME_CACH_CME_LINEBUF_W, cols));
}
+/*
+ * HEVC motion estimation: the same search window as H.264 (+-44 x +-28
+ * pixels). The CIME cache depends on the tile, see vepu580_setup_hevc_cime().
+ */
+void vepu580_setup_hevc_me(struct rkvenc_core *core, unsigned int width,
+ unsigned int height, bool colmv_stor, bool colmv_load)
+{
+ unsigned int width_mbs = DIV_ROUND_UP(width, 16);
+ unsigned int height_mbs = DIV_ROUND_UP(height, 16);
+ unsigned int blk_w = 44, blk_h = 28, hal;
+
+ if (blk_w / 4 * 2 > (width_mbs * 2 + 1) / 2)
+ blk_w = (width_mbs * 2 + 1) / 2 / 2 * 4;
+
+ hal = ALIGN(height_mbs * 16, 64);
+ if (blk_h / 4 > hal / 128 * 4)
+ blk_h = hal / 128 * 16;
+
+ rkvenc_set(core, VEPU580_ME_RNGE,
+ FIELD_PREP(VEPU580_ME_RNGE_CME_SRCH_H, blk_w / 4) |
+ FIELD_PREP(VEPU580_ME_RNGE_CME_SRCH_V, blk_h / 4) |
+ FIELD_PREP(VEPU580_ME_RNGE_RME_SRCH_H, 7) |
+ FIELD_PREP(VEPU580_ME_RNGE_RME_SRCH_V, 5) |
+ FIELD_PREP(VEPU580_ME_RNGE_DLT_FRM_NUM, 1));
+
+ rkvenc_set(core, VEPU580_ME_CFG,
+ FIELD_PREP(VEPU580_ME_CFG_PMV_MDST_H, 5) |
+ FIELD_PREP(VEPU580_ME_CFG_PMV_MDST_V, 5) |
+ FIELD_PREP(VEPU580_ME_CFG_PMV_NUM, 2) |
+ (colmv_stor ? VEPU580_ME_CFG_COLMV_STOR : 0) |
+ (colmv_load ? VEPU580_ME_CFG_COLMV_LOAD : 0));
+}
+
/*
* Scale a lambda table entry: base[] holds three (or six) consecutive
* entries starting at QP @first, and the table doubles every @period QPs.
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
index 61c745994da9..a82a22137488 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
@@ -757,6 +757,33 @@ static void rkvenc_read_feedback(struct rkvenc_core *core,
fb->cycles = readl_relaxed(r + VEPU580_DBG_FRAME_CYC);
}
+/* Add the status of one hardware pass to that of the earlier ones. */
+static void rkvenc_add_feedback(struct rkvenc_feedback *sum,
+ const struct rkvenc_feedback *fb, bool first)
+{
+ unsigned int i;
+
+ if (first) {
+ *sum = *fb;
+ return;
+ }
+
+ sum->int_sta |= fb->int_sta;
+ sum->bs_len += fb->bs_len;
+ sum->sse += fb->sse;
+ sum->qp_sum += fb->qp_sum;
+ sum->madi += fb->madi;
+ sum->madp += fb->madp;
+ sum->intra_blocks += fb->intra_blocks;
+ for (i = 0; i < RKVENC_PNUM_COUNT; i++)
+ sum->pnum[i] += fb->pnum[i];
+ if (fb->qp_min >= 0 && (sum->qp_min < 0 || fb->qp_min < sum->qp_min))
+ sum->qp_min = fb->qp_min;
+ if (fb->qp_max > sum->qp_max)
+ sum->qp_max = fb->qp_max;
+ sum->cycles += fb->cycles;
+}
+
static void rkvenc_release_core(struct rkvenc_core *core)
{
struct rkvenc_dev *rkvenc = core->rkvenc;
@@ -850,6 +877,8 @@ static void rkvenc_device_run(void *priv)
core->run.src = src;
core->run.dst = dst;
core->run.hdr_bytes = 0;
+ core->run.pass = 0;
+ memset(&core->run.fb, 0, sizeof(core->run.fb));
req = src->vb2_buf.req_obj.req;
if (req)
@@ -925,6 +954,25 @@ static irqreturn_t rkvenc_irq_thread(int irq, void *priv)
dev_err(core->dev, "encode error, irq status %#x\n", sta);
state = VB2_BUF_STATE_ERROR;
rkvenc_hw_reset(core);
+ } else {
+ const struct rkvenc_codec_ops *ops =
+ core->curr_ctx->coded_fmt_desc->ops;
+ int ret = 0;
+
+ rkvenc_add_feedback(&core->run.fb, &fb, !core->run.pass);
+
+ /* Pictures encoded in several passes, e.g. HEVC tile columns */
+ if (ops->next_pass)
+ ret = ops->next_pass(core->curr_ctx, core, &core->run);
+ if (ret > 0) {
+ core->run.pass++;
+ rkvenc_hw_start(core);
+ pm_runtime_put_noidle(core->dev);
+ return IRQ_HANDLED;
+ }
+ if (ret < 0)
+ state = VB2_BUF_STATE_ERROR;
+ fb = core->run.fb;
}
pm_runtime_put_noidle(core->dev);
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
index ec6a955921b8..c1dce59d8308 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
@@ -98,6 +98,10 @@ struct rkvenc_run {
struct vb2_v4l2_buffer *dst;
/* Bytes written by software at the start of @dst (parameter sets). */
unsigned int hdr_bytes;
+ /* Hardware pass of the picture, counted from 0. */
+ unsigned int pass;
+ /* Status read-back of the finished passes, summed. */
+ struct rkvenc_feedback fb;
};
/**
@@ -107,6 +111,11 @@ struct rkvenc_run {
* @stop: called on STREAMOFF, frees codec state
* @run: prepare the job: produce software headers and fill @core's shadow
* registers (codec syntax, rate control and references)
+ * @next_pass: optional, called after each hardware pass that finished without
+ * error, with the status of the passes so far summed in @run->fb.
+ * Returns 1 after updating @core's shadow registers for another
+ * pass of the same picture, 0 when the picture is complete, or a
+ * negative error code.
* @done: called after the hardware finished; @fb holds the status read-back.
* The backend sets the CAPTURE payload and may change @state.
*/
@@ -116,6 +125,8 @@ struct rkvenc_codec_ops {
void (*stop)(struct rkvenc_ctx *ctx);
int (*run)(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
struct rkvenc_run *run);
+ int (*next_pass)(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
+ struct rkvenc_run *run);
void (*done)(struct rkvenc_ctx *ctx, struct rkvenc_run *run,
const struct rkvenc_feedback *fb,
enum vb2_buffer_state *state);
@@ -130,10 +141,13 @@ struct rkvenc_ctrls {
* struct rkvenc_coded_fmt_desc - coded (CAPTURE) format description
* @size_align: alignment of the encoded picture size (enc_rsl), the
* hardware pads the source up to it
+ * @ext_line_buf: pictures wider than 4096 pixels are encoded in one pass
+ * with the external line buffer
*/
struct rkvenc_coded_fmt_desc {
u32 fourcc;
unsigned int size_align;
+ bool ext_line_buf;
struct v4l2_frmsize_stepwise frmsize;
const struct rkvenc_ctrls *ctrls;
const struct rkvenc_codec_ops *ops;
@@ -209,10 +223,19 @@ struct rkvenc_h264_ctx {
/**
* struct rkvenc_hevc_ctx - HEVC per-context state
+ * @enc: HEVC stateless encode core
+ * @hw_rc: the current picture uses the hardware row rate control
+ * @tile_buf: loop filter data at the tile column boundaries, one part of
+ * @tile_buf_size bytes per tile column (pictures wider than 4096)
+ * @tile_buf_size: size of one part of @tile_buf
+ * @tile: tile column the current hardware pass encodes
*/
struct rkvenc_hevc_ctx {
struct v4l2_hevc_enc enc;
bool hw_rc;
+ struct rkvenc_aux_buf tile_buf;
+ size_t tile_buf_size;
+ unsigned int tile;
};
struct rkvenc_ctx {
@@ -247,6 +270,12 @@ static inline void rkvenc_or(struct rkvenc_core *core, u32 reg, u32 val)
core->shadow[reg / 4] |= val;
}
+static inline void rkvenc_update(struct rkvenc_core *core, u32 reg, u32 mask,
+ u32 val)
+{
+ core->shadow[reg / 4] = (core->shadow[reg / 4] & ~mask) | (val & mask);
+}
+
/* Shared VEPU580 programming helpers (rkvenc-vepu580.c). */
int rkvenc_alloc_aux(struct rkvenc_dev *rkvenc, struct rkvenc_aux_buf *buf,
size_t size, bool kernel_mapping);
@@ -262,6 +291,8 @@ void vepu580_setup_h264_tuning(struct rkvenc_dev *rkvenc,
struct rkvenc_core *core, bool intra);
void vepu580_setup_hevc_me(struct rkvenc_core *core, unsigned int width,
unsigned int height, bool colmv_stor, bool colmv_load);
+void vepu580_setup_hevc_cime(struct rkvenc_core *core, unsigned int width,
+ unsigned int tile_x, unsigned int tile_w);
void vepu580_setup_hevc_tuning(struct rkvenc_core *core, bool intra);
/**
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 09/12] media: rockchip: rkvenc: Limit the QP of the HEVC I pictures to 15..51
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (7 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 08/12] media: rockchip: rkvenc: Encode HEVC wider than 4096 pixels in tile columns Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 10/12] media: rockchip: rkvenc: Re-encode oversized HEVC pictures Arun Iyengar via B4 Relay
` (2 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
As the Rockchip MPP library does by default for HEVC, keep the I pictures
of the frame rate control at QP 15 or above, while the P pictures may go
down to 10: add the I picture QP limit controls, with the defaults 15 and
51. The hardware row rate control of an I picture gets the same limits.
Move the QP setup of the picture to a function of its own.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
.../media/platform/rockchip/rkvenc/rkvenc-hevc.c | 61 +++++++++++++++-------
1 file changed, 41 insertions(+), 20 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
index 27006e393121..9baf76f7fc43 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
@@ -567,6 +567,43 @@ static void rkvenc_hevc_setup_tile(struct rkvenc_hevc_ctx *hevc,
buf + (tile ? tile - 1 : 1) * hevc->tile_buf_size);
}
+/*
+ * QP of the picture. In CBR mode the size target of the frame rate control
+ * is handed to the hardware CTU row rate control, which adapts the QP of
+ * each CTU row by up to the row_rc_qp_range parameter, within the QP limits
+ * of the picture type.
+ */
+static void rkvenc_hevc_setup_qp(struct rkvenc_hevc_ctx *hevc,
+ struct rkvenc_core *core)
+{
+ struct v4l2_hevc_enc *enc = &hevc->enc;
+ const struct v4l2_hevc_enc_state *state = &enc->state_active;
+ bool intra = state->encode.slice_type == V4L2_HEVC_SLICE_TYPE_I;
+ unsigned int qp_range;
+ int qp_min, qp_max;
+
+ rkvenc_update(core, VEPU580_ENC_PIC, VEPU580_ENC_PIC_PIC_QP, 0);
+ rkvenc_update(core, VEPU580_HEVC_SYNT_SLI1, VEPU580_HEVC_SLI1_SLI_QP,
+ FIELD_PREP(VEPU580_HEVC_SLI1_SLI_QP, enc->qp));
+
+ qp_range = min(READ_ONCE(rkvenc_row_rc_qp_range), 15U);
+ hevc->hw_rc = state->frame_rc_enable &&
+ state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR &&
+ qp_range;
+ if (!hevc->hw_rc) {
+ vepu580_setup_fixed_qp(core, enc->qp);
+ return;
+ }
+
+ qp_min = intra ? state->qp_min_i : state->qp_min;
+ qp_max = intra ? state->qp_max_i : state->qp_max;
+ vepu580_setup_row_rc(core, &rkvenc_hevc_row_rc, enc->qp,
+ clamp(qp_min, 0, 51), clamp(qp_max, 0, 51),
+ qp_range, enc->cbr.target_bits,
+ state->width_ctbs * state->height_ctbs,
+ state->width_ctbs);
+}
+
static int rkvenc_hevc_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
struct rkvenc_run *run)
{
@@ -574,7 +611,6 @@ static int rkvenc_hevc_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
struct v4l2_hevc_enc *enc = &hevc->enc;
const struct v4l2_hevc_enc_state *state = &enc->state_active;
struct rkvenc_rec_buf *rec, *refr = NULL, *col = NULL;
- unsigned int ctus, qp_range;
bool intra, tmvp, colmv_stor;
int ret;
@@ -591,7 +627,6 @@ static int rkvenc_hevc_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
return -ENOSPC;
intra = state->encode.slice_type == V4L2_HEVC_SLICE_TYPE_I;
- ctus = state->width_ctbs * state->height_ctbs;
rec = enc->ref.buffer_current.private_data;
if (!rec)
@@ -645,24 +680,7 @@ static int rkvenc_hevc_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
colmv_stor, col && col->colmv_valid);
vepu580_setup_hevc_tuning(core, intra);
- /*
- * In CBR mode the size target of the frame rate control is handed to
- * the hardware CTU row rate control, which adapts the QP of each CTU
- * row by up to the row_rc_qp_range parameter.
- */
- qp_range = min(READ_ONCE(rkvenc_row_rc_qp_range), 15U);
- hevc->hw_rc = state->frame_rc_enable &&
- state->bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR &&
- qp_range;
- if (hevc->hw_rc)
- vepu580_setup_row_rc(core, &rkvenc_hevc_row_rc, enc->qp,
- clamp(state->qp_min, 0, 51),
- clamp(state->qp_max, 0, 51), qp_range,
- enc->cbr.target_bits, ctus,
- state->width_ctbs);
- else
- vepu580_setup_fixed_qp(core, enc->qp);
-
+ rkvenc_hevc_setup_qp(hevc, core);
rkvenc_hevc_setup_tile(hevc, core, 0);
return 0;
@@ -807,6 +825,9 @@ static const struct v4l2_ctrl_config rkvenc_hevc_ctrl_cfgs[] = {
},
{ .id = V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP, .max = 51, .step = 1, .def = 10, },
{ .id = V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP, .max = 51, .step = 1, .def = 51, },
+ /* I pictures of the frame rate control: 15..51 as in MPP */
+ { .id = V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MIN_QP, .max = 51, .step = 1, .def = 15, },
+ { .id = V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MAX_QP, .max = 51, .step = 1, .def = 51, },
{ .id = V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP, .max = 51, .step = 1, .def = 30, },
{ .id = V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP, .max = 51, .step = 1, .def = 32, },
{ .id = V4L2_CID_MPEG_VIDEO_AVERAGE_QP, .max = 51, .step = 1, },
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 10/12] media: rockchip: rkvenc: Re-encode oversized HEVC pictures
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (8 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 09/12] media: rockchip: rkvenc: Limit the QP of the HEVC I pictures to 15..51 Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 11/12] media: rockchip: rkvenc: Offer VBR for H.264 encoding Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 12/12] arm64: dts: rockchip: Add the VEPU580 encoder cores to the RK3588 Arun Iyengar via B4 Relay
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
Let the constant bitrate mode of the HEVC encode core code a picture far
above its size target a second time, as the Rockchip MPP library does:
once the last pass of the picture finished, ask v4l2_hevc_enc_reencode()
and run the whole picture again with the new QP, writing the slice data
over that of the first pass. The hardware writes the slice header, so the
software headers stay as they are.
A codec backend now returns RKVENC_PASS_AGAIN from next_pass to encode the
whole picture again, which restarts the summed status of the passes, and
RKVENC_PASS_NEXT for the next pass of the same picture.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
.../media/platform/rockchip/rkvenc/rkvenc-hevc.c | 24 +++++++++++++++++-----
drivers/media/platform/rockchip/rkvenc/rkvenc.c | 6 +++++-
drivers/media/platform/rockchip/rkvenc/rkvenc.h | 11 +++++++---
3 files changed, 32 insertions(+), 9 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
index 9baf76f7fc43..acbbfdd3f613 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
@@ -688,19 +688,33 @@ static int rkvenc_hevc_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
/*
* The next tile column continues the bitstream right after the previous
- * one, as a slice of its own (the hardware writes its slice header).
+ * one, as a slice of its own (the hardware writes its slice header). Once
+ * the picture is complete, the rate control may have it encoded again with
+ * a higher QP, from the first tile column, over the slice data of the first
+ * encode.
*/
static int rkvenc_hevc_next_pass(struct rkvenc_ctx *ctx,
struct rkvenc_core *core,
struct rkvenc_run *run)
{
struct rkvenc_hevc_ctx *hevc = ctx->priv;
- const struct v4l2_ctrl_hevc_pps *pps = &hevc->enc.state_active.pps;
+ struct v4l2_hevc_enc *enc = &hevc->enc;
+ const struct v4l2_ctrl_hevc_pps *pps = &enc->state_active.pps;
u64 offset = run->hdr_bytes + run->fb.bs_len;
if (!(pps->flags & V4L2_HEVC_PPS_FLAG_TILES_ENABLED) ||
- hevc->tile >= pps->num_tile_columns_minus1)
- return 0;
+ hevc->tile >= pps->num_tile_columns_minus1) {
+ if (!run->fb.bs_len || offset > rkvenc_dst_size(run) ||
+ !v4l2_hevc_enc_reencode(enc, offset))
+ return 0;
+
+ rkvenc_set(core, VEPU580_ADR_BSBS,
+ rkvenc_dst_addr(run) + run->hdr_bytes);
+ rkvenc_hevc_setup_qp(hevc, core);
+ rkvenc_hevc_setup_tile(hevc, core, 0);
+
+ return RKVENC_PASS_AGAIN;
+ }
if (!run->fb.bs_len || offset + SZ_4K > rkvenc_dst_size(run))
return -ENOSPC;
@@ -708,7 +722,7 @@ static int rkvenc_hevc_next_pass(struct rkvenc_ctx *ctx,
rkvenc_set(core, VEPU580_ADR_BSBS, rkvenc_dst_addr(run) + offset);
rkvenc_hevc_setup_tile(hevc, core, hevc->tile + 1);
- return 1;
+ return RKVENC_PASS_NEXT;
}
static void rkvenc_hevc_done(struct rkvenc_ctx *ctx, struct rkvenc_run *run,
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
index a82a22137488..7313523af6b3 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c
@@ -965,7 +965,11 @@ static irqreturn_t rkvenc_irq_thread(int irq, void *priv)
if (ops->next_pass)
ret = ops->next_pass(core->curr_ctx, core, &core->run);
if (ret > 0) {
- core->run.pass++;
+ /* A picture encoded again starts a new status sum. */
+ if (ret == RKVENC_PASS_AGAIN)
+ core->run.pass = 0;
+ else
+ core->run.pass++;
rkvenc_hw_start(core);
pm_runtime_put_noidle(core->dev);
return IRQ_HANDLED;
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
index c1dce59d8308..8fae10e2c69d 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc.h
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h
@@ -113,12 +113,17 @@ struct rkvenc_run {
* registers (codec syntax, rate control and references)
* @next_pass: optional, called after each hardware pass that finished without
* error, with the status of the passes so far summed in @run->fb.
- * Returns 1 after updating @core's shadow registers for another
- * pass of the same picture, 0 when the picture is complete, or a
- * negative error code.
+ * Returns RKVENC_PASS_NEXT after updating @core's shadow registers
+ * for another pass of the same picture, RKVENC_PASS_AGAIN after
+ * setting them up to encode the whole picture again (the status
+ * of the passes so far is dropped), 0 when the picture is
+ * complete, or a negative error code.
* @done: called after the hardware finished; @fb holds the status read-back.
* The backend sets the CAPTURE payload and may change @state.
*/
+#define RKVENC_PASS_NEXT 1
+#define RKVENC_PASS_AGAIN 2
+
struct rkvenc_codec_ops {
int (*adjust_fmt)(struct rkvenc_ctx *ctx, struct v4l2_format *f);
int (*start)(struct rkvenc_ctx *ctx);
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 11/12] media: rockchip: rkvenc: Offer VBR for H.264 encoding
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (9 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 10/12] media: rockchip: rkvenc: Re-encode oversized HEVC pictures Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
2026-11-10 10:08 ` [PATCH RFC 12/12] arm64: dts: rockchip: Add the VEPU580 encoder cores to the RK3588 Arun Iyengar via B4 Relay
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The H.264 encode core has a variable bitrate mode now. Offer it in
V4L2_CID_MPEG_VIDEO_BITRATE_MODE and add V4L2_CID_MPEG_VIDEO_BITRATE_PEAK,
which the core reads for it (default 20 Mbit/s, twice the default
bitrate), and hand the size target of the variable bitrate pictures to the
hardware row rate control as for constant bitrate.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
index fcb8c070ab30..313c33038670 100644
--- a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
+++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
@@ -381,11 +381,13 @@ static int rkvenc_h264_run(struct rkvenc_ctx *ctx, struct rkvenc_core *core,
vepu580_setup_h264_tuning(ctx->dev, core, intra);
/*
- * In CBR mode the frame size target of the rate control is handed to
- * the hardware row rate control, which adapts the QP of each
+ * In CBR and VBR mode the frame size target of the rate control is
+ * handed to the hardware row rate control, which adapts the QP of each
* macroblock row by up to the row_rc_qp_range parameter.
*/
- h264->hw_rc = rc->mode && rc->mode->type == V4L2_H264_ENC_RC_TYPE_CBR &&
+ h264->hw_rc = rc->mode &&
+ (rc->mode->type == V4L2_H264_ENC_RC_TYPE_CBR ||
+ rc->mode->type == V4L2_H264_ENC_RC_TYPE_VBR) &&
READ_ONCE(rkvenc_row_rc_qp_range);
if (h264->hw_rc)
vepu580_setup_row_rc(core, &vepu580_h264_row_rc, rc->qp,
@@ -466,12 +468,15 @@ static const struct v4l2_ctrl_config rkvenc_h264_ctrl_cfgs[] = {
.min = V4L2_MPEG_VIDEO_BITRATE_MODE_VBR,
.max = V4L2_MPEG_VIDEO_BITRATE_MODE_CQ,
.def = V4L2_MPEG_VIDEO_BITRATE_MODE_CBR,
- .menu_skip_mask = BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_VBR),
},
{
.id = V4L2_CID_MPEG_VIDEO_BITRATE,
.min = 10000, .max = 200000000, .step = 1, .def = 10000000,
},
+ {
+ .id = V4L2_CID_MPEG_VIDEO_BITRATE_PEAK,
+ .min = 10000, .max = 200000000, .step = 1, .def = 20000000,
+ },
{
.id = V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY,
.min = 1, .max = 100, .step = 1, .def = 50,
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH RFC 12/12] arm64: dts: rockchip: Add the VEPU580 encoder cores to the RK3588
2026-11-10 10:08 [PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588 Arun Iyengar via B4 Relay
` (10 preceding siblings ...)
2026-11-10 10:08 ` [PATCH RFC 11/12] media: rockchip: rkvenc: Offer VBR for H.264 encoding Arun Iyengar via B4 Relay
@ 2026-11-10 10:08 ` Arun Iyengar via B4 Relay
11 siblings, 0 replies; 13+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:08 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Heiko Stuebner, Philipp Zabel
Cc: linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, Paul Kocialkowski, Sven Püschel,
Nicolas Dufresne, Detlev Casanova, Hans Verkuil, Jiaxing Hu,
Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
Add the two VEPU580 (RKVENC) video encoder cores and their IOMMUs.
Interrupts, clocks, resets, power domains and the clock rates (500 MHz
AXI, 800 MHz core) are taken from the Rockchip BSP rk3588s.dtsi. Each
IOMMU has two register banks and two interrupts, and keeps the BSP's
rockchip,disable-mmu-reset.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
arch/arm64/boot/dts/rockchip/rk3588-base.dtsi | 56 +++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
diff --git a/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi b/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi
index aa69fd3793fd..9c813e6475c5 100644
--- a/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3588-base.dtsi
@@ -1397,6 +1397,62 @@ vepu121_3_mmu: iommu@fdbac800 {
#iommu-cells = <0>;
};
+ vepu580_0: video-codec@fdbd0000 {
+ compatible = "rockchip,rk3588-vepu580";
+ reg = <0x0 0xfdbd0000 0x0 0x6000>;
+ interrupts = <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_RKVENC0>, <&cru HCLK_RKVENC0>,
+ <&cru CLK_RKVENC0_CORE>;
+ clock-names = "aclk", "hclk", "core";
+ assigned-clocks = <&cru ACLK_RKVENC0>, <&cru CLK_RKVENC0_CORE>;
+ assigned-clock-rates = <500000000>, <800000000>;
+ resets = <&cru SRST_A_RKVENC0>, <&cru SRST_H_RKVENC0>,
+ <&cru SRST_RKVENC0_CORE>;
+ reset-names = "aclk", "hclk", "core";
+ iommus = <&vepu580_0_mmu>;
+ power-domains = <&power RK3588_PD_VENC0>;
+ };
+
+ vepu580_0_mmu: iommu@fdbdf000 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdbdf000 0x0 0x40>, <0x0 0xfdbdf040 0x0 0x40>;
+ interrupts = <GIC_SPI 99 IRQ_TYPE_LEVEL_HIGH 0>,
+ <GIC_SPI 100 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_RKVENC0>, <&cru HCLK_RKVENC0>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VENC0>;
+ rockchip,disable-mmu-reset;
+ #iommu-cells = <0>;
+ };
+
+ vepu580_1: video-codec@fdbe0000 {
+ compatible = "rockchip,rk3588-vepu580";
+ reg = <0x0 0xfdbe0000 0x0 0x6000>;
+ interrupts = <GIC_SPI 104 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_RKVENC1>, <&cru HCLK_RKVENC1>,
+ <&cru CLK_RKVENC1_CORE>;
+ clock-names = "aclk", "hclk", "core";
+ assigned-clocks = <&cru ACLK_RKVENC1>, <&cru CLK_RKVENC1_CORE>;
+ assigned-clock-rates = <500000000>, <800000000>;
+ resets = <&cru SRST_A_RKVENC1>, <&cru SRST_H_RKVENC1>,
+ <&cru SRST_RKVENC1_CORE>;
+ reset-names = "aclk", "hclk", "core";
+ iommus = <&vepu580_1_mmu>;
+ power-domains = <&power RK3588_PD_VENC1>;
+ };
+
+ vepu580_1_mmu: iommu@fdbef000 {
+ compatible = "rockchip,rk3588-iommu", "rockchip,rk3568-iommu";
+ reg = <0x0 0xfdbef000 0x0 0x40>, <0x0 0xfdbef040 0x0 0x40>;
+ interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH 0>,
+ <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH 0>;
+ clocks = <&cru ACLK_RKVENC1>, <&cru HCLK_RKVENC1>;
+ clock-names = "aclk", "iface";
+ power-domains = <&power RK3588_PD_VENC1>;
+ rockchip,disable-mmu-reset;
+ #iommu-cells = <0>;
+ };
+
vdec0: video-codec@fdc38000 {
compatible = "rockchip,rk3588-vdec";
reg = <0x0 0xfdc38000 0x0 0x100>,
--
2.55.0
^ permalink raw reply [flat|nested] 13+ messages in thread