* [PATCH RFC 01/11] media: h264: Only write the SPS and PPS of a stateless encode when they change
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 02/11] media: h264: Support picture order count type 2 in stateless encode Arun Iyengar via B4 Relay
` (9 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The stateless encode core is meant to write the SPS and PPS for the first
picture of a stream, and then only when they change. The check is
inverted: the first picture compares the new SPS and PPS with the zeroed
active state, which always differs, and all the following pictures get
them unconditionally. This wastes some bits on every picture, and skews
the size statistics of the rate control.
Compare with the active state once a picture was encoded, and write them
unconditionally for the first picture. The
V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR control still forces them on
IDR pictures.
To be squashed into "media: h264: Add stateless encode core" from
https://patch.msgid.link/20260522101653.2565125-5-paulk@sys-base.io
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-h264-enc.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc.c b/drivers/media/v4l2-core/v4l2-h264-enc.c
index 67ea82cebe7f..af925b6626cf 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc.c
@@ -762,9 +762,12 @@ static int rbsp_update(struct v4l2_h264_enc *enc)
if (ctrl && ctrl->cur.val)
enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_AUD;
- /* SPS */
+ /*
+ * SPS: always written for the first picture of the stream, then only
+ * when it changes.
+ */
- if (!enc->state_serial) {
+ if (enc->state_serial) {
if (memcmp(&state_active->sps, &state->sps,
sizeof(state_active->sps)) ||
memcmp(&state_active->sps_video, &state->sps_video,
@@ -774,9 +777,9 @@ static int rbsp_update(struct v4l2_h264_enc *enc)
enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_SPS;
}
- /* PPS */
+ /* PPS: same as the SPS. */
- if (!enc->state_serial) {
+ if (enc->state_serial) {
if (memcmp(&state_active->pps, &state->pps,
sizeof(state_active->pps)))
enc->rbsp_update |= V4L2_H264_ENC_RBSP_UPDATE_PPS;
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 02/11] media: h264: Support picture order count type 2 in stateless encode
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 01/11] media: h264: Only write the SPS and PPS of a stateless encode when they change Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 03/11] media: h264: Write the stateless encode SPS and PPS with nal_ref_idc 3 Arun Iyengar via B4 Relay
` (8 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
Some encoders write the slice header in hardware and can't put a
pic_order_cnt_lsb in it: the Hantro H1 and Rockchip VEPU1/VEPU2 encoders
only produce slice headers for pic_order_cnt_type 2, where the picture
order count is derived from frame_num.
Accept pic_order_cnt_type 2, which drivers can then select in their
state_constrain operation, and compute the picture order count as
specified in ITU-T H.264 section 8.2.1.3. This needs the frame_num and
FrameNumOffset of the previous picture in decoding order, which are now
tracked for every picture, reference or not.
Type 2 implies that the output order is the decoding order, which is
always the case without B frames. Type 1 is still rejected.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c | 3 +-
drivers/media/v4l2-core/v4l2-h264-enc.c | 51 +++++++++++++++++++++++++---
include/media/v4l2-h264-enc.h | 3 ++
3 files changed, 52 insertions(+), 5 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c b/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
index a165f8114ce9..2d751855195b 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
@@ -758,7 +758,8 @@ int v4l2_h264_enc_rbsp_sps(struct v4l2_h264_enc_rbsp *rbsp,
if (ret)
return ret;
}
- } else {
+ } else if (sps->pic_order_cnt_type != 2) {
+ /* Type 2 has no further syntax elements. */
return -EINVAL;
}
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc.c b/drivers/media/v4l2-core/v4l2-h264-enc.c
index af925b6626cf..9745c0e85ee5 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc.c
@@ -200,10 +200,11 @@ static int state_prepare_params(struct v4l2_h264_enc *enc)
sps->pic_height_in_map_units_minus1 = state->height_mbs - 1;
/*
- * Only pic_order_cnt_type = 0 is currently supported.
- * Error out since no pic_order_cnt_lsb was provided.
+ * Only pic_order_cnt_type 0 and 2 are currently supported. Type 2
+ * derives the picture order count from frame_num, for hardware that
+ * can't write pic_order_cnt_lsb in the slice header.
*/
- if (sps->pic_order_cnt_type)
+ if (sps->pic_order_cnt_type == 1)
return -EINVAL;
/* SPS Video */
@@ -933,6 +934,41 @@ static int ref_step_buffers(struct v4l2_h264_enc *enc)
return 0;
}
+/* Picture order count type 2, see ITU-T H.264 section 8.2.1.3. */
+static int ref_step_poc_type2(struct v4l2_h264_enc *enc)
+{
+ struct v4l2_h264_enc_state *state = &enc->state_active;
+ struct v4l2_ctrl_h264_sps *sps = &state->sps;
+ struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
+ struct v4l2_h264_enc_ref *ref = &enc->ref;
+ unsigned int max_frame_num = BIT(sps->log2_max_frame_num_minus4 + 4);
+ unsigned int frame_num_offset;
+ unsigned int pic_order_cnt;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC)
+ frame_num_offset = 0;
+ else if (ref->prev_frame_num > encode->frame_num)
+ frame_num_offset = ref->prev_frame_num_offset + max_frame_num;
+ else
+ frame_num_offset = ref->prev_frame_num_offset;
+
+ if (encode->flags & V4L2_H264_ENCODE_FLAG_IDR_PIC)
+ pic_order_cnt = 0;
+ else if (!encode->nal_ref_idc)
+ pic_order_cnt = 2 * (frame_num_offset + encode->frame_num) - 1;
+ else
+ pic_order_cnt = 2 * (frame_num_offset + encode->frame_num);
+
+ ref->frame_num_offset = frame_num_offset;
+ ref->pic_order_cnt_msb = 0;
+ ref->pic_order_cnt_lsb = 0;
+ ref->top_field_order_cnt = pic_order_cnt;
+ ref->bottom_field_order_cnt = pic_order_cnt;
+ ref->pic_order_cnt = pic_order_cnt;
+
+ return 0;
+}
+
static int ref_step_poc(struct v4l2_h264_enc *enc)
{
struct v4l2_h264_enc_state *state = &enc->state_active;
@@ -949,7 +985,10 @@ static int ref_step_poc(struct v4l2_h264_enc *enc)
unsigned int top_field_order_cnt;
unsigned int bottom_field_order_cnt;
- /* Only pic_order_cnt_type = 0 is currently supported. */
+ if (sps->pic_order_cnt_type == 2)
+ return ref_step_poc_type2(enc);
+
+ /* Only pic_order_cnt_type 0 and 2 are currently supported. */
if (sps->pic_order_cnt_type)
return -EINVAL;
@@ -1174,6 +1213,10 @@ static int ref_complete(struct v4l2_h264_enc *enc,
unsigned int index;
int ret;
+ /* Keep the frame number of the previous picture for POC type 2. */
+ ref->prev_frame_num = encode->frame_num;
+ ref->prev_frame_num_offset = ref->frame_num_offset;
+
if (!encode->nal_ref_idc)
return 0;
diff --git a/include/media/v4l2-h264-enc.h b/include/media/v4l2-h264-enc.h
index 6debdc95232c..ff7166109551 100644
--- a/include/media/v4l2-h264-enc.h
+++ b/include/media/v4l2-h264-enc.h
@@ -54,6 +54,9 @@ struct v4l2_h264_enc_ref {
struct v4l2_h264_reflist_builder builder;
+ unsigned int prev_frame_num;
+ unsigned int prev_frame_num_offset;
+ unsigned int frame_num_offset;
unsigned int prev_pic_order_cnt_msb;
unsigned int prev_pic_order_cnt_lsb;
unsigned int pic_order_cnt_msb;
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 03/11] media: h264: Write the stateless encode SPS and PPS with nal_ref_idc 3
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 01/11] media: h264: Only write the SPS and PPS of a stateless encode when they change Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 02/11] media: h264: Support picture order count type 2 in stateless encode Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 04/11] media: v4l2-h264-enc-rc: Rework the constant bitrate mode Arun Iyengar via B4 Relay
` (7 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The sequence and picture parameter set NAL units are written with
nal_ref_idc 0, which ITU-T H.264 section 7.4.1 forbids: "nal_ref_idc
shall not be equal to 0 for sequence parameter set or sequence parameter
set extension or subset sequence parameter set or picture parameter set
NAL units". FFmpeg's H.264 probe rejects such streams (an Annex B file
from the encoder is "detected only with low score of 1"), and stricter
decoders may drop the parameter sets.
Write them with nal_ref_idc 3, as other encoders do. Access unit
delimiters keep 0, which the specification requires for them.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c b/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
index 2d751855195b..fa5857b77d76 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rbsp.c
@@ -654,7 +654,7 @@ int v4l2_h264_enc_rbsp_sps(struct v4l2_h264_enc_rbsp *rbsp,
unsigned int i;
int ret;
- ret = v4l2_h264_enc_rbsp_nalu_begin(rbsp, 0, V4L2_H264_NALU_TYPE_SPS);
+ ret = v4l2_h264_enc_rbsp_nalu_begin(rbsp, 3, V4L2_H264_NALU_TYPE_SPS);
if (ret)
return ret;
@@ -845,7 +845,7 @@ int v4l2_h264_enc_rbsp_pps(struct v4l2_h264_enc_rbsp *rbsp,
u8 pic_scaling_matrix_present_flag = 0;
int ret;
- ret = v4l2_h264_enc_rbsp_nalu_begin(rbsp, 0, V4L2_H264_NALU_TYPE_PPS);
+ ret = v4l2_h264_enc_rbsp_nalu_begin(rbsp, 3, V4L2_H264_NALU_TYPE_PPS);
if (ret)
return ret;
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 04/11] media: v4l2-h264-enc-rc: Rework the constant bitrate mode
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
` (2 preceding siblings ...)
2026-11-10 10:05 ` [PATCH RFC 03/11] media: h264: Write the stateless encode SPS and PPS with nal_ref_idc 3 Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 05/11] media: v4l2-core: Move the H.264 encode CBR rate control to a helper Arun Iyengar via B4 Relay
` (6 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The CBR mode of the H.264 encode core misses its bitrate and has a large
size spike after each IDR picture:
- the QP only moves when a picture misses its size target by more than
25% (the ratio table has a dead band from 0.8 to 1.25), so a steady
error below that is never corrected and the QP stays near its start
value;
- P pictures get 0.8 times the average size left in a 50 picture budget,
which settles at about 0.89 times the nominal picture size: the
bitrate stays 5-9% low;
- I and P pictures keep separate QP histories, so an IDR picture is coded
with the I QP of the previous GOP, seconds old, and the first P
picture after it uses the P history against a reference of a very
different quality, and overshoots its target several times;
- the start QP comes from a fixed bits per macroblock table and a P
target 6 times below the I table.
Replace it with a model like the frame-level rate control of the Rockchip
MPP library: one QP state for the stream (the QP of the P pictures), I
pictures 2 QP below it, a start QP from the bits per luma sample of the
first I picture, picture size targets from a GOP plan with the I/P size
ratio and the IDR period measured on the stream, and the deviation of
the coded size from that plan spread over the next 8 pictures. After
each picture the QP moves by 1/8 of its QP error, 6 * log2(size /
target), and by half of the part beyond 3 QP steps.
The constant QP and constant quality modes are not changed.
Board results (RK3588S Hantro H1, 1080p60 clips of 300 pictures, GOP 120,
2-16 Mb/s, GStreamer v4l2slh264enc): the bitrate error goes from
-8.3..+2.3% to +0.3..+2.3% (-0.0..+0.2% over whole GOPs), PSNR-Y
improves by 0.2-0.6 dB, the first P picture after an IDR goes from up to
20 times to at most 1.4 times the median P size, and the encode speed is
unchanged.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 426 +++++++++++++++--------------
include/media/v4l2-h264-enc-rc.h | 34 +++
2 files changed, 248 insertions(+), 212 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
index 8f9622395eb6..bbfe03febac8 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
@@ -5,6 +5,9 @@
* Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
*/
+#include <linux/int_log.h>
+#include <linux/math64.h>
+#include <linux/minmax.h>
#include <linux/module.h>
#include <media/v4l2-h264-enc.h>
@@ -158,275 +161,274 @@ static const struct v4l2_h264_enc_rc_mode cq_mode = {
.estimate = cq_estimate,
};
-static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
-{
- struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
- struct v4l2_fract *timeperframe = &state->timeperframe;
- struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
- unsigned long long budget;
- unsigned long target;
- unsigned long target_nominal;
-
- pr_debug("+ v4l2-h264-enc-rc: cbr measure");
-
- /* Nominal average size target for exact bitrate. */
- target_nominal = state->bitrate * timeperframe->numerator /
- timeperframe->denominator / 8;
-
- /* Total size budget for twice the size window. */
- budget = 2 * V4L2_H264_ENC_RC_STATS_SIZE_COUNT * state->bitrate *
- timeperframe->numerator / timeperframe->denominator / 8;
-
- if (budget < stats->size_total) {
- target = target_nominal;
- goto complete;
- }
+/*
+ * Constant bitrate mode
+ *
+ * One QP state is kept for the stream: it is the QP of the P (and B)
+ * pictures, and I pictures are coded a fixed number of steps below it, so
+ * that the quality of the reference pictures stays even across GOPs. Each
+ * picture gets a size target from a GOP plan, in which an I picture is
+ * "ratio" times the size of a P picture, with the ratio and the IDR period
+ * measured on the stream. The difference between the coded size so far and
+ * the size the plan expects at this point of the GOP is spread over the next
+ * pictures, so that the bitrate converges to the target.
+ *
+ * After each picture the QP state moves by a fraction of the QP error
+ * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
+ * errors, so that the QP does not follow the size noise from picture to
+ * picture, and quickly for errors of several QP steps (a scene change, a
+ * poor start).
+ */
- /* Remaining size budget after deducing total size in window. */
- budget -= stats->size_total;
- /* Average size target from remaining size budget. */
- target = budget / (2 * V4L2_H264_ENC_RC_STATS_SIZE_COUNT - stats->count);
+/* The accumulated size error is spread over this many pictures. */
+#define CBR_WINDOW 8
+/* Limit of the accumulated size error, in seconds of the bitrate */
+#define CBR_BUFFER_SECONDS 3
+/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
+#define CBR_RATIO_INIT 160
+#define CBR_RATIO_MIN 24
+#define CBR_RATIO_MAX 480
+/* P pictures after an I picture when the I/P size ratio is measured */
+#define CBR_RATIO_PICS 4
+/* QP of the I pictures below the QP of the P pictures */
+#define CBR_QP_DELTA_INTRA 2
+/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
+#define CBR_ERR_SLOW (3 * 256)
+/*
+ * Largest QP increase and decrease per picture, 1/256 units. Decreases are
+ * slower, so that pictures that are much smaller than their target, such as
+ * repeated pictures, don't pull the QP down far.
+ */
+#define CBR_STEP_UP_MAX (4 * 256)
+#define CBR_STEP_DOWN_MAX (2 * 256)
-complete:
- pr_debug(" nominal target: %lu bytes", target_nominal);
- pr_debug(" uniform target: %lu bytes", target);
+/* log2(x) in 8.8 fixed point */
+static int cbr_log2_q8(u64 x)
+{
+ unsigned int shift = 0;
- /* Bump size for intra frames and reduce for pred frames.
- * We are better off with good quality initial references.
- * Maintain average assuming 1 intra frame for 24 inter frames.
- */
- if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
- target = target * 6;
- else
- target = target * 80 / 100;
+ if (!x)
+ return 0;
- /* Reduce target size on bitrate overshoot to calm things down. */
- if (9 * stats->size_total > 10 * stats->count * target_nominal) {
- pr_debug(" overshoot, reducing target size");
- target = 80 * target / 100;
+ while (x > U32_MAX) {
+ x >>= 1;
+ shift++;
}
- pr_debug(" final target: %lu bytes", target);
-
- *size = target;
-
- return 0;
+ return (int)(intlog2(x) >> 16) + shift * 256;
}
-/* Intra bit size per macroblock estimation for QP starting at 10. */
-static unsigned int cbr_qp_initial_bits_mb[] = {
- 437, 411, 376, 355, 318, 303, 291, 271, 252, 238, 218, 204, 188, 171,
- 158, 149, 133, 124, 115, 105, 98, 92, 83, 77, 70, 63, 57, 52, 45, 39,
- 34, 31, 30, 29, 26, 23, 22, 20, 18, 17, 16, 15
-};
-
-static int cbr_qp_initial(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
{
- struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
- unsigned long size_mb;
- unsigned int qp_initial;
- unsigned int threshold;
- unsigned int i;
+ if (!timeperframe->numerator || !timeperframe->denominator)
+ return 30;
- size_mb = 8 * rc->size / (state->width_mbs * state->height_mbs);
-
- pr_debug(" initial estimate from %lu bits/mb", size_mb);
+ return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
+ timeperframe->numerator), 1U);
+}
- for (i = 0; i < ARRAY_SIZE(cbr_qp_initial_bits_mb); i++) {
- threshold = cbr_qp_initial_bits_mb[i];
+static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
+{
+ if (!timeperframe->numerator || !timeperframe->denominator)
+ return max(bitrate / 30, 1U);
- /* Expect inter frames to be 6 times smaller. */
- if (encode->slice_type != V4L2_H264_SLICE_TYPE_I)
- threshold = cbr_qp_initial_bits_mb[i] / 6;
+ return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
+ timeperframe->denominator), 1);
+}
- if (size_mb >= threshold) {
- qp_initial = 10 + i;
- break;
- }
- }
+/* Planned size of an I or P picture of the current GOP */
+static u64 cbr_plan(const struct v4l2_h264_enc_rc_cbr *cbr, unsigned int fps,
+ bool intra)
+{
+ unsigned int gop = cbr->gop;
+ u64 p;
- /* Fallback to the lowest QP (last in the list). */
- if (i == ARRAY_SIZE(cbr_qp_initial_bits_mb))
- qp_initial = 10 + i - 1;
+ /* Until the second I picture, assume at least two seconds. */
+ if (!gop)
+ gop = max(cbr->since_i + 1, 2 * fps);
+ if (gop <= 1)
+ return cbr->nominal;
- *qp = qp_initial;
+ p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
+ if (intra)
+ p = div_u64(p * cbr->ratio, 16);
- return 0;
+ return max_t(u64, p, 1);
}
/*
- * This table gives the QP decrease/increase (coded as index, with QP = 0 in
- * the middle) for each size ratio (in base 1000).
- *
- * Intra is a bit agressive since we generally have few intra slices to adapt
- * to a scene change. The QP diff range goes from -4 to +4.
- */
-static unsigned long cbr_ratio_qp_diff_steps_intra[] = {
- 3000, 2000, 1500, 1250, 800, 666, 500, 333
-};
-
-/*
- * Inter is less agressive since we generally have more inter slices to adapt
- * to a scene change and want to give more stability to QP. The QP diff range
- * goes from -3 to +3.
+ * Start QP of an I picture from its size in bits per luma sample:
+ * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
+ * the Rockchip MPP library uses for its first picture at 1080p.
*/
-static unsigned long cbr_ratio_qp_diff_steps_inter[] = {
- 2000, 1500, 1250, 800, 666, 500
-};
-
-static int cbr_ratio_qp_diff(unsigned long ratio, unsigned char slice_type)
+static int cbr_qp_initial(u64 bits, unsigned int samples)
{
- unsigned long *steps;
- unsigned int count;
- unsigned int i;
-
- if (slice_type == V4L2_H264_SLICE_TYPE_I) {
- steps = cbr_ratio_qp_diff_steps_intra;
- count = ARRAY_SIZE(cbr_ratio_qp_diff_steps_intra);
- } else {
- steps = cbr_ratio_qp_diff_steps_inter;
- count = ARRAY_SIZE(cbr_ratio_qp_diff_steps_inter);
- }
+ int qp_q8 = 20 * 256 + 77 +
+ 6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
- WARN_ON(count % 2);
+ return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
+}
- for (i = 0; i < count; i++)
- if (ratio > steps[i])
- return -1 * (int)count / 2 + i;
+static int cbr_init(struct v4l2_h264_enc_rc *rc)
+{
+ memset(&rc->cbr, 0, sizeof(rc->cbr));
- return count / 2;
+ return 0;
}
-static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
{
struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_ctrl_h264_encode_params *encode = &state->encode;
- struct v4l2_h264_enc_rc_stats *stats = &rc->stats;
- struct v4l2_h264_enc_rc_stats_type *type;
- unsigned int type_index[3];
- unsigned int ratio_qp[3];
- unsigned int penalty[3];
- unsigned int penalty_total;
- unsigned int weight[3];
- unsigned int weight_total;
- int ratio_qp_diff;
- unsigned long diff;
- unsigned long diff_closest;
- unsigned int slot_closest;
- unsigned int index;
- unsigned int count;
- unsigned int total;
- unsigned long ratio;
- unsigned int i;
-
- if (!stats->count)
- return cbr_qp_initial(rc, qp);
-
- if (encode->slice_type == V4L2_H264_SLICE_TYPE_I)
- type = &stats->intra;
- else if (encode->slice_type == V4L2_H264_SLICE_TYPE_P)
- type = &stats->pred;
- else if (encode->slice_type == V4L2_H264_SLICE_TYPE_B)
- type = &stats->bipred;
- else
+ struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+ struct v4l2_fract *timeperframe = &state->timeperframe;
+ bool intra = state->encode.slice_type == V4L2_H264_SLICE_TYPE_I;
+ unsigned int fps = cbr_fps(timeperframe);
+ u64 nominal = cbr_nominal(state->bitrate, timeperframe);
+ unsigned int samples = state->width_mbs * state->height_mbs * 256;
+ s64 target, expected = 0;
+ int qp;
+
+ if (!state->bitrate)
return -EINVAL;
- pr_debug("+ v4l2-h264-enc-rc: cbr estimate");
-
- if (!type->count)
- return cbr_qp_initial(rc, qp);
-
- for (i = 0; i < 3; i++) {
- if (i == type->count)
- break;
-
- index = type->index;
- if (index <= i)
- index = type->count + index - (i + 1);
- else
- index -= i + 1;
-
- /* Set initial penalty based on frame age. */
- penalty[i] = 4 * i;
- type_index[i] = index;
-
- /* Track frame with size closest to target. */
- diff = abs((long)rc->size - type->size[index]);
-
- if (!i || diff < diff_closest) {
- diff_closest = diff;
- slot_closest = i;
- }
+ if (!cbr->bitrate) {
+ cbr->ratio = CBR_RATIO_INIT;
+ cbr->qp_q8 = -1;
}
- count = i;
- penalty_total = 0;
-
- for (i = 0; i < count; i++) {
- /* Add penalty for frames with larger size difference. */
- if (i != slot_closest)
- penalty[i] += 10;
+ /* A new bitrate keeps the QP state and the measured statistics. */
+ if (cbr->bitrate != state->bitrate || cbr->nominal != nominal) {
+ cbr->bitrate = state->bitrate;
+ cbr->nominal = nominal;
+ cbr->buffer_bits = 0;
+ }
- penalty_total += penalty[i];
+ if (intra) {
+ /* An I picture closes the GOP: its length is the IDR period. */
+ if (cbr->seen_i)
+ cbr->gop = cbr->since_i + 1;
+ cbr->since_i = 0;
+ cbr->seen_i = true;
+ }
- index = type_index[i];
+ cbr->plan_bits = cbr_plan(cbr, fps, intra);
- /* Calculate QP from target to observed size ratio. */
- ratio = 1000UL * rc->size / type->size[index];
+ if (cbr->qp_q8 < 0) {
+ /* First picture: start QP from the bits per sample. */
+ qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
+ CBR_QP_DELTA_INTRA;
+ cbr->qp_q8 = clamp_t(int, qp, state->qp_min, state->qp_max) *
+ 256;
+ }
- ratio_qp_diff = cbr_ratio_qp_diff(ratio, encode->slice_type);
+ /*
+ * The buffer holds the coded minus the nominal sizes. Within a GOP the
+ * plan expects it to go up by the excess of the I picture and back
+ * down to zero by the savings of the P pictures until the next I
+ * picture. Spread the difference to that over the next pictures.
+ */
+ if (!intra && cbr->seen_i)
+ expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
+ (s64)cbr->since_i *
+ ((s64)cbr->plan_bits - (s64)nominal);
+
+ target = (s64)cbr->plan_bits -
+ div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
+ CBR_WINDOW);
+ cbr->target_bits = clamp_t(s64, target,
+ max_t(u64, cbr->plan_bits / 4, 8),
+ cbr->plan_bits * 2);
+ cbr->intra = intra;
- if ((int)type->qp[index] + ratio_qp_diff < 0)
- ratio_qp[i] = 0;
- else if ((int)type->qp[index] + ratio_qp_diff > 51)
- ratio_qp[i] = 51;
- else
- ratio_qp[i] = type->qp[index] + ratio_qp_diff;
+ pr_debug("+ v4l2-h264-enc-rc: cbr measure");
+ pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
+ cbr->nominal, cbr->plan_bits, cbr->target_bits);
+ pr_debug(" buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
+ cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
- diff = abs((long)rc->size - type->size[index]);
+ *size = max_t(unsigned long, cbr->target_bits / 8, 1);
- pr_debug(" - backlog %d size: %lu (diff: %lu), qp: %u, penalty %u, ratio qp: %u",
- -(i + 1), type->size[index], diff, type->qp[index],
- penalty[i], ratio_qp[i]);
- }
+ return 0;
+}
- total = 0;
- weight_total = 0;
+static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
+{
+ struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+ int value;
- for (i = 0; i < count; i++) {
- /* Weight each calculated QP using the penalty ratio. */
- if (penalty_total)
- weight[i] = 1000 * (penalty_total - penalty[i]) /
- penalty_total;
- else
- weight[i] = 1000;
+ value = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
+ if (cbr->intra)
+ value -= CBR_QP_DELTA_INTRA;
- /* Give non-zero low weight to full penalty cases. */
- if (weight[i] < 200)
- weight[i] = 200;
+ *qp = clamp(value, 0, 51);
- total += ratio_qp[i] * weight[i];
- weight_total += weight[i];
+ return 0;
+}
- pr_debug(" - backlog %d weight: %u", -(i + 1),
- weight[i]);
+static void cbr_update(struct v4l2_h264_enc_rc *rc, u64 bits)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+ struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
+ s64 limit = cbr->nominal * CBR_BUFFER_SECONDS *
+ cbr_fps(&state->timeperframe);
+ unsigned int ratio;
+ int err, step;
+
+ if (!cbr->bitrate || !cbr->target_bits)
+ return;
+
+ cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
+ (s64)cbr->nominal, -limit, limit);
+
+ /* QP error of the picture, 1/256 units */
+ err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
+
+ step = err / 8;
+ if (err > CBR_ERR_SLOW)
+ step += (err - CBR_ERR_SLOW) / 2;
+ else if (err < -CBR_ERR_SLOW)
+ step += (err + CBR_ERR_SLOW) / 2;
+ step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
+
+ if (!cbr->intra) {
+ cbr->qp_q8 += step;
+ cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
+ cbr->since_i++;
+
+ /* I/P size ratio, once a few P pictures were coded */
+ if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
+ ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
+ ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
+ cbr->ratio = (cbr->ratio + ratio + 1) / 2;
+ }
+ } else {
+ /*
+ * The size of an I picture mostly reflects the I/P ratio
+ * estimate: only all-intra streams steer the QP fully with it.
+ */
+ cbr->i_bits = bits;
+ cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
}
- *qp = total / weight_total;
+ cbr->qp_q8 = clamp_t(int, cbr->qp_q8, state->qp_min * 256,
+ state->qp_max * 256);
- return 0;
+ pr_debug("+ v4l2-h264-enc-rc: cbr update");
+ pr_debug(" size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
+ bits, cbr->target_bits, err, cbr->qp_q8);
}
static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
{
+ cbr_update(rc, (u64)bytesused * 8);
+
return v4l2_h264_enc_rc_stats_collect(rc, bytesused);
}
static const struct v4l2_h264_enc_rc_mode cbr_mode = {
.name = "constant bitrate",
.type = V4L2_H264_ENC_RC_TYPE_CBR,
+ .init = cbr_init,
.measure = cbr_measure,
.estimate = cbr_estimate,
.complete = cbr_complete,
diff --git a/include/media/v4l2-h264-enc-rc.h b/include/media/v4l2-h264-enc-rc.h
index 8b453c3a212e..72d78a594e40 100644
--- a/include/media/v4l2-h264-enc-rc.h
+++ b/include/media/v4l2-h264-enc-rc.h
@@ -8,6 +8,7 @@
#ifndef _MEDIA_V4L2_H264_ENC_RC_H
#define _MEDIA_V4L2_H264_ENC_RC_H
+#include <linux/types.h>
#include <linux/v4l2-controls.h>
#include <media/v4l2-h264.h>
@@ -82,6 +83,38 @@ struct v4l2_h264_enc_rc_stats {
unsigned int index;
};
+/**
+ * struct v4l2_h264_enc_rc_cbr - constant bitrate mode state
+ * @bitrate: bitrate the state was set up for, 0 before the first picture
+ * @nominal: nominal picture size in bits (bitrate times frame interval)
+ * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
+ * @ratio: I/P picture size ratio in 1/16 units
+ * @gop: measured IDR period in pictures, 0 until two I pictures were seen
+ * @since_i: pictures coded since the last I picture
+ * @seen_i: an I picture was coded
+ * @intra: the current picture is an I picture
+ * @p_bits: running mean of the P picture sizes
+ * @i_bits: size of the last I picture
+ * @buffer_bits: accumulated difference between the coded and nominal sizes
+ * @plan_bits: planned size of the current picture
+ * @target_bits: size target of the current picture
+ */
+struct v4l2_h264_enc_rc_cbr {
+ u32 bitrate;
+ u64 nominal;
+ int qp_q8;
+ unsigned int ratio;
+ unsigned int gop;
+ unsigned int since_i;
+ bool seen_i;
+ bool intra;
+ u64 p_bits;
+ u64 i_bits;
+ s64 buffer_bits;
+ u64 plan_bits;
+ u64 target_bits;
+};
+
struct v4l2_h264_enc_rc {
const struct v4l2_h264_enc_rc_ops *ops;
void *private_data;
@@ -91,6 +124,7 @@ struct v4l2_h264_enc_rc {
void *mode_data;
struct v4l2_h264_enc_rc_stats stats;
+ struct v4l2_h264_enc_rc_cbr cbr;
unsigned long size;
unsigned int qp;
};
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 05/11] media: v4l2-core: Move the H.264 encode CBR rate control to a helper
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
` (3 preceding siblings ...)
2026-11-10 10:05 ` [PATCH RFC 04/11] media: v4l2-h264-enc-rc: Rework the constant bitrate mode Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 06/11] media: v4l2-core: Bound the I picture size in the encode CBR rate control Arun Iyengar via B4 Relay
` (5 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The constant bitrate mode of the H.264 encode core does not depend on the
codec: it plans picture sizes over the GOP from the measured I/P size
ratio and IDR period, and moves one QP state by the QP error 6 *
log2(size / target), which holds for HEVC as well. Only the QP delta of
the I pictures differs between the codecs.
Move it to v4l2-enc-cbr, with the I picture QP delta as a parameter, so
that the HEVC encode core can use the same model. The H.264 core keeps
its mode callbacks and uses a QP delta of 2, as before. No functional
change.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/Kconfig | 5 +
drivers/media/v4l2-core/Makefile | 1 +
drivers/media/v4l2-core/v4l2-enc-cbr.c | 300 +++++++++++++++++++++++++++++
drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 246 ++---------------------
include/media/v4l2-enc-cbr.h | 64 ++++++
include/media/v4l2-h264-enc-rc.h | 35 +---
6 files changed, 387 insertions(+), 264 deletions(-)
diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index a88eaf8af48c..b8b849a44abd 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -44,9 +44,14 @@ config V4L2_JPEG_HELPER
config V4L2_H264
tristate
+# Used by the stateless encode cores that need v4l2-enc-cbr.ko
+config V4L2_ENC_CBR
+ tristate
+
# Used by drivers that need v4l2-h264-enc.ko
config V4L2_H264_ENC
tristate
+ select V4L2_ENC_CBR
# Used by drivers that need v4l2-vp9.ko
config V4L2_VP9
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index 81c5582d6a34..3aa0855d7be7 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -26,6 +26,7 @@ videodev-$(CONFIG_VIDEO_V4L2_I2C) += v4l2-i2c.o
obj-$(CONFIG_V4L2_ASYNC) += v4l2-async.o
obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
+obj-$(CONFIG_V4L2_ENC_CBR) += v4l2-enc-cbr.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
diff --git a/drivers/media/v4l2-core/v4l2-enc-cbr.c b/drivers/media/v4l2-core/v4l2-enc-cbr.c
new file mode 100644
index 000000000000..5bd10d494054
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-enc-cbr.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * V4L2 stateless encoder constant bitrate rate control
+ *
+ * One QP state is kept for the stream: it is the QP of the P (and B)
+ * pictures, and I pictures are coded a fixed number of steps below it, so
+ * that the quality of the reference pictures stays even across GOPs. Each
+ * picture gets a size target from a GOP plan, in which an I picture is
+ * "ratio" times the size of a P picture, with the ratio and the IDR period
+ * measured on the stream. The difference between the coded size so far and
+ * the size the plan expects at this point of the GOP is spread over the next
+ * pictures, so that the bitrate converges to the target.
+ *
+ * After each picture the QP state moves by a fraction of the QP error
+ * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
+ * errors, so that the QP does not follow the size noise from picture to
+ * picture, and quickly for errors of several QP steps (a scene change, a
+ * poor start).
+ *
+ * The QP to size relation (6 QP steps per halving) is the same for H.264
+ * and HEVC, so the model does not depend on the codec. Only the QP delta of
+ * the I pictures is a parameter.
+ */
+
+#include <linux/export.h>
+#include <linux/int_log.h>
+#include <linux/math64.h>
+#include <linux/minmax.h>
+#include <linux/module.h>
+#include <linux/printk.h>
+#include <linux/string.h>
+#include <media/v4l2-enc-cbr.h>
+
+/* The accumulated size error is spread over this many pictures. */
+#define CBR_WINDOW 8
+/* Limit of the accumulated size error, in seconds of the bitrate */
+#define CBR_BUFFER_SECONDS 3
+/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
+#define CBR_RATIO_INIT 160
+#define CBR_RATIO_MIN 24
+#define CBR_RATIO_MAX 480
+/* P pictures after an I picture when the I/P size ratio is measured */
+#define CBR_RATIO_PICS 4
+/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
+#define CBR_ERR_SLOW (3 * 256)
+/*
+ * Largest QP increase and decrease per picture, 1/256 units. Decreases are
+ * slower, so that pictures that are much smaller than their target, such as
+ * repeated pictures, don't pull the QP down far.
+ */
+#define CBR_STEP_UP_MAX (4 * 256)
+#define CBR_STEP_DOWN_MAX (2 * 256)
+
+/* log2(x) in 8.8 fixed point */
+static int cbr_log2_q8(u64 x)
+{
+ unsigned int shift = 0;
+
+ if (!x)
+ return 0;
+
+ while (x > U32_MAX) {
+ x >>= 1;
+ shift++;
+ }
+
+ return (int)(intlog2(x) >> 16) + shift * 256;
+}
+
+static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
+{
+ if (!timeperframe->numerator || !timeperframe->denominator)
+ return 30;
+
+ return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
+ timeperframe->numerator), 1U);
+}
+
+static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
+{
+ if (!timeperframe->numerator || !timeperframe->denominator)
+ return max(bitrate / 30, 1U);
+
+ return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
+ timeperframe->denominator), 1);
+}
+
+/* Planned size of an I or P picture of the current GOP */
+static u64 cbr_plan(const struct v4l2_enc_cbr *cbr, unsigned int fps,
+ bool intra)
+{
+ unsigned int gop = cbr->gop;
+ u64 p;
+
+ /* Until the second I picture, assume at least two seconds. */
+ if (!gop)
+ gop = max(cbr->since_i + 1, 2 * fps);
+ if (gop <= 1)
+ return cbr->nominal;
+
+ p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
+ if (intra)
+ p = div_u64(p * cbr->ratio, 16);
+
+ return max_t(u64, p, 1);
+}
+
+/*
+ * Start QP of an I picture from its size in bits per luma sample:
+ * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
+ * the Rockchip MPP library uses for its first picture at 1080p, for H.264
+ * and HEVC alike.
+ */
+static int cbr_qp_initial(u64 bits, unsigned int samples)
+{
+ int qp_q8 = 20 * 256 + 77 +
+ 6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
+
+ return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
+}
+
+/**
+ * v4l2_enc_cbr_reset() - reset the rate control state
+ * @cbr: the rate control state
+ * @qp_delta_intra: QP of the I pictures below the QP of the P pictures
+ *
+ * The next picture starts the rate control afresh.
+ */
+void v4l2_enc_cbr_reset(struct v4l2_enc_cbr *cbr, int qp_delta_intra)
+{
+ memset(cbr, 0, sizeof(*cbr));
+ cbr->qp_delta_intra = qp_delta_intra;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_reset);
+
+/**
+ * v4l2_enc_cbr_begin() - set up the size target of the next picture
+ * @cbr: the rate control state
+ * @bitrate: target bitrate in bits per second
+ * @timeperframe: frame interval
+ * @samples: number of luma samples of a picture
+ * @intra: the picture is an I picture
+ * @qp_min: lowest QP
+ * @qp_max: highest QP
+ *
+ * Sets @cbr->target_bits. The QP of the picture is then given by
+ * v4l2_enc_cbr_qp(). A change of the bitrate keeps the QP state and the
+ * measured statistics.
+ *
+ * Return: 0, or -EINVAL if @bitrate is 0.
+ */
+int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
+ const struct v4l2_fract *timeperframe,
+ unsigned int samples, bool intra, int qp_min,
+ int qp_max)
+{
+ unsigned int fps = cbr_fps(timeperframe);
+ u64 nominal = cbr_nominal(bitrate, timeperframe);
+ s64 target, expected = 0;
+ int qp;
+
+ if (!bitrate)
+ return -EINVAL;
+
+ if (!cbr->bitrate) {
+ cbr->ratio = CBR_RATIO_INIT;
+ cbr->qp_q8 = -1;
+ }
+
+ if (cbr->bitrate != bitrate || cbr->nominal != nominal) {
+ cbr->bitrate = bitrate;
+ cbr->nominal = nominal;
+ cbr->buffer_bits = 0;
+ }
+
+ if (intra) {
+ /* An I picture closes the GOP: its length is the IDR period. */
+ if (cbr->seen_i)
+ cbr->gop = cbr->since_i + 1;
+ cbr->since_i = 0;
+ cbr->seen_i = true;
+ }
+
+ cbr->plan_bits = cbr_plan(cbr, fps, intra);
+
+ if (cbr->qp_q8 < 0) {
+ /* First picture: start QP from the bits per sample. */
+ qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
+ cbr->qp_delta_intra;
+ cbr->qp_q8 = clamp_t(int, qp, qp_min, qp_max) * 256;
+ }
+
+ /*
+ * The buffer holds the coded minus the nominal sizes. Within a GOP the
+ * plan expects it to go up by the excess of the I picture and back
+ * down to zero by the savings of the P pictures until the next I
+ * picture. Spread the difference to that over the next pictures.
+ */
+ if (!intra && cbr->seen_i)
+ expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
+ (s64)cbr->since_i *
+ ((s64)cbr->plan_bits - (s64)nominal);
+
+ target = (s64)cbr->plan_bits -
+ div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
+ CBR_WINDOW);
+ cbr->target_bits = clamp_t(s64, target,
+ max_t(u64, cbr->plan_bits / 4, 8),
+ cbr->plan_bits * 2);
+ cbr->intra = intra;
+
+ pr_debug("+ v4l2-enc-cbr: begin");
+ pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
+ cbr->nominal, cbr->plan_bits, cbr->target_bits);
+ pr_debug(" buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
+ cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_begin);
+
+/**
+ * v4l2_enc_cbr_qp() - QP of the picture set up by v4l2_enc_cbr_begin()
+ * @cbr: the rate control state
+ *
+ * Return: the QP, not limited to the QP range of the codec.
+ */
+int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr)
+{
+ int qp = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
+
+ if (cbr->intra)
+ qp -= cbr->qp_delta_intra;
+
+ return qp;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_qp);
+
+/**
+ * v4l2_enc_cbr_end() - account for a coded picture
+ * @cbr: the rate control state
+ * @bits: size of the coded picture in bits
+ * @timeperframe: frame interval
+ * @qp_min: lowest QP
+ * @qp_max: highest QP
+ */
+void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
+ const struct v4l2_fract *timeperframe, int qp_min,
+ int qp_max)
+{
+ s64 limit = cbr->nominal * CBR_BUFFER_SECONDS * cbr_fps(timeperframe);
+ unsigned int ratio;
+ int err, step;
+
+ if (!cbr->bitrate || !cbr->target_bits)
+ return;
+
+ cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
+ (s64)cbr->nominal, -limit, limit);
+
+ /* QP error of the picture, 1/256 units */
+ err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
+
+ step = err / 8;
+ if (err > CBR_ERR_SLOW)
+ step += (err - CBR_ERR_SLOW) / 2;
+ else if (err < -CBR_ERR_SLOW)
+ step += (err + CBR_ERR_SLOW) / 2;
+ step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
+
+ if (!cbr->intra) {
+ cbr->qp_q8 += step;
+ cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
+ cbr->since_i++;
+
+ /* I/P size ratio, once a few P pictures were coded */
+ if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
+ ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
+ ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
+ cbr->ratio = (cbr->ratio + ratio + 1) / 2;
+ }
+ } else {
+ /*
+ * The size of an I picture mostly reflects the I/P ratio
+ * estimate: only all-intra streams steer the QP fully with it.
+ */
+ cbr->i_bits = bits;
+ cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
+ }
+
+ cbr->qp_q8 = clamp_t(int, cbr->qp_q8, qp_min * 256, qp_max * 256);
+
+ pr_debug("+ v4l2-enc-cbr: end");
+ pr_debug(" size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
+ bits, cbr->target_bits, err, cbr->qp_q8);
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_end);
+
+MODULE_DESCRIPTION("V4L2 stateless encoder constant bitrate rate control");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
index bbfe03febac8..c4f1718ead0e 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
@@ -5,7 +5,6 @@
* Copyright (C) 2025-2026 Paul Kocialkowski <paulk@sys-base.io>
*/
-#include <linux/int_log.h>
#include <linux/math64.h>
#include <linux/minmax.h>
#include <linux/module.h>
@@ -162,116 +161,15 @@ static const struct v4l2_h264_enc_rc_mode cq_mode = {
};
/*
- * Constant bitrate mode
- *
- * One QP state is kept for the stream: it is the QP of the P (and B)
- * pictures, and I pictures are coded a fixed number of steps below it, so
- * that the quality of the reference pictures stays even across GOPs. Each
- * picture gets a size target from a GOP plan, in which an I picture is
- * "ratio" times the size of a P picture, with the ratio and the IDR period
- * measured on the stream. The difference between the coded size so far and
- * the size the plan expects at this point of the GOP is spread over the next
- * pictures, so that the bitrate converges to the target.
- *
- * After each picture the QP state moves by a fraction of the QP error
- * 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
- * errors, so that the QP does not follow the size noise from picture to
- * picture, and quickly for errors of several QP steps (a scene change, a
- * poor start).
+ * Constant bitrate mode, with the codec independent rate control of
+ * v4l2-enc-cbr: I pictures 2 QP below the P pictures (the default of the
+ * Rockchip MPP library for H.264).
*/
-
-/* The accumulated size error is spread over this many pictures. */
-#define CBR_WINDOW 8
-/* Limit of the accumulated size error, in seconds of the bitrate */
-#define CBR_BUFFER_SECONDS 3
-/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
-#define CBR_RATIO_INIT 160
-#define CBR_RATIO_MIN 24
-#define CBR_RATIO_MAX 480
-/* P pictures after an I picture when the I/P size ratio is measured */
-#define CBR_RATIO_PICS 4
-/* QP of the I pictures below the QP of the P pictures */
#define CBR_QP_DELTA_INTRA 2
-/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
-#define CBR_ERR_SLOW (3 * 256)
-/*
- * Largest QP increase and decrease per picture, 1/256 units. Decreases are
- * slower, so that pictures that are much smaller than their target, such as
- * repeated pictures, don't pull the QP down far.
- */
-#define CBR_STEP_UP_MAX (4 * 256)
-#define CBR_STEP_DOWN_MAX (2 * 256)
-
-/* log2(x) in 8.8 fixed point */
-static int cbr_log2_q8(u64 x)
-{
- unsigned int shift = 0;
-
- if (!x)
- return 0;
-
- while (x > U32_MAX) {
- x >>= 1;
- shift++;
- }
-
- return (int)(intlog2(x) >> 16) + shift * 256;
-}
-
-static unsigned int cbr_fps(const struct v4l2_fract *timeperframe)
-{
- if (!timeperframe->numerator || !timeperframe->denominator)
- return 30;
-
- return max(DIV_ROUND_CLOSEST(timeperframe->denominator,
- timeperframe->numerator), 1U);
-}
-
-static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
-{
- if (!timeperframe->numerator || !timeperframe->denominator)
- return max(bitrate / 30, 1U);
-
- return max_t(u64, div_u64((u64)bitrate * timeperframe->numerator,
- timeperframe->denominator), 1);
-}
-
-/* Planned size of an I or P picture of the current GOP */
-static u64 cbr_plan(const struct v4l2_h264_enc_rc_cbr *cbr, unsigned int fps,
- bool intra)
-{
- unsigned int gop = cbr->gop;
- u64 p;
-
- /* Until the second I picture, assume at least two seconds. */
- if (!gop)
- gop = max(cbr->since_i + 1, 2 * fps);
- if (gop <= 1)
- return cbr->nominal;
-
- p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
- if (intra)
- p = div_u64(p * cbr->ratio, 16);
-
- return max_t(u64, p, 1);
-}
-
-/*
- * Start QP of an I picture from its size in bits per luma sample:
- * 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
- * the Rockchip MPP library uses for its first picture at 1080p.
- */
-static int cbr_qp_initial(u64 bits, unsigned int samples)
-{
- int qp_q8 = 20 * 256 + 77 +
- 6 * (cbr_log2_q8(samples) - cbr_log2_q8(max_t(u64, bits, 1)));
-
- return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
-}
static int cbr_init(struct v4l2_h264_enc_rc *rc)
{
- memset(&rc->cbr, 0, sizeof(rc->cbr));
+ v4l2_enc_cbr_reset(&rc->cbr, CBR_QP_DELTA_INTRA);
return 0;
}
@@ -279,148 +177,34 @@ static int cbr_init(struct v4l2_h264_enc_rc *rc)
static int cbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
{
struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- struct v4l2_fract *timeperframe = &state->timeperframe;
bool intra = state->encode.slice_type == V4L2_H264_SLICE_TYPE_I;
- unsigned int fps = cbr_fps(timeperframe);
- u64 nominal = cbr_nominal(state->bitrate, timeperframe);
unsigned int samples = state->width_mbs * state->height_mbs * 256;
- s64 target, expected = 0;
- int qp;
-
- if (!state->bitrate)
- return -EINVAL;
-
- if (!cbr->bitrate) {
- cbr->ratio = CBR_RATIO_INIT;
- cbr->qp_q8 = -1;
- }
-
- /* A new bitrate keeps the QP state and the measured statistics. */
- if (cbr->bitrate != state->bitrate || cbr->nominal != nominal) {
- cbr->bitrate = state->bitrate;
- cbr->nominal = nominal;
- cbr->buffer_bits = 0;
- }
-
- if (intra) {
- /* An I picture closes the GOP: its length is the IDR period. */
- if (cbr->seen_i)
- cbr->gop = cbr->since_i + 1;
- cbr->since_i = 0;
- cbr->seen_i = true;
- }
-
- cbr->plan_bits = cbr_plan(cbr, fps, intra);
+ int ret;
- if (cbr->qp_q8 < 0) {
- /* First picture: start QP from the bits per sample. */
- qp = cbr_qp_initial(cbr_plan(cbr, fps, true), samples) +
- CBR_QP_DELTA_INTRA;
- cbr->qp_q8 = clamp_t(int, qp, state->qp_min, state->qp_max) *
- 256;
- }
+ ret = v4l2_enc_cbr_begin(&rc->cbr, state->bitrate,
+ &state->timeperframe, samples, intra,
+ state->qp_min, state->qp_max);
+ if (ret)
+ return ret;
- /*
- * The buffer holds the coded minus the nominal sizes. Within a GOP the
- * plan expects it to go up by the excess of the I picture and back
- * down to zero by the savings of the P pictures until the next I
- * picture. Spread the difference to that over the next pictures.
- */
- if (!intra && cbr->seen_i)
- expected = (s64)cbr_plan(cbr, fps, true) - (s64)nominal +
- (s64)cbr->since_i *
- ((s64)cbr->plan_bits - (s64)nominal);
-
- target = (s64)cbr->plan_bits -
- div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
- CBR_WINDOW);
- cbr->target_bits = clamp_t(s64, target,
- max_t(u64, cbr->plan_bits / 4, 8),
- cbr->plan_bits * 2);
- cbr->intra = intra;
-
- pr_debug("+ v4l2-h264-enc-rc: cbr measure");
- pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
- cbr->nominal, cbr->plan_bits, cbr->target_bits);
- pr_debug(" buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
- cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
-
- *size = max_t(unsigned long, cbr->target_bits / 8, 1);
+ *size = max_t(unsigned long, rc->cbr.target_bits / 8, 1);
return 0;
}
static int cbr_estimate(struct v4l2_h264_enc_rc *rc, unsigned int *qp)
{
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- int value;
-
- value = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
- if (cbr->intra)
- value -= CBR_QP_DELTA_INTRA;
-
- *qp = clamp(value, 0, 51);
+ *qp = clamp(v4l2_enc_cbr_qp(&rc->cbr), 0, 51);
return 0;
}
-static void cbr_update(struct v4l2_h264_enc_rc *rc, u64 bits)
+static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
{
struct v4l2_h264_enc_state *state = rc->state;
- struct v4l2_h264_enc_rc_cbr *cbr = &rc->cbr;
- s64 limit = cbr->nominal * CBR_BUFFER_SECONDS *
- cbr_fps(&state->timeperframe);
- unsigned int ratio;
- int err, step;
-
- if (!cbr->bitrate || !cbr->target_bits)
- return;
-
- cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
- (s64)cbr->nominal, -limit, limit);
-
- /* QP error of the picture, 1/256 units */
- err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
-
- step = err / 8;
- if (err > CBR_ERR_SLOW)
- step += (err - CBR_ERR_SLOW) / 2;
- else if (err < -CBR_ERR_SLOW)
- step += (err + CBR_ERR_SLOW) / 2;
- step = clamp(step, -CBR_STEP_DOWN_MAX, CBR_STEP_UP_MAX);
-
- if (!cbr->intra) {
- cbr->qp_q8 += step;
- cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
- cbr->since_i++;
-
- /* I/P size ratio, once a few P pictures were coded */
- if (cbr->since_i == CBR_RATIO_PICS && cbr->i_bits) {
- ratio = div64_u64(cbr->i_bits * 16, cbr->p_bits);
- ratio = clamp(ratio, CBR_RATIO_MIN, CBR_RATIO_MAX);
- cbr->ratio = (cbr->ratio + ratio + 1) / 2;
- }
- } else {
- /*
- * The size of an I picture mostly reflects the I/P ratio
- * estimate: only all-intra streams steer the QP fully with it.
- */
- cbr->i_bits = bits;
- cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
- }
- cbr->qp_q8 = clamp_t(int, cbr->qp_q8, state->qp_min * 256,
- state->qp_max * 256);
-
- pr_debug("+ v4l2-h264-enc-rc: cbr update");
- pr_debug(" size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
- bits, cbr->target_bits, err, cbr->qp_q8);
-}
-
-static int cbr_complete(struct v4l2_h264_enc_rc *rc, unsigned long bytesused)
-{
- cbr_update(rc, (u64)bytesused * 8);
+ v4l2_enc_cbr_end(&rc->cbr, (u64)bytesused * 8, &state->timeperframe,
+ state->qp_min, state->qp_max);
return v4l2_h264_enc_rc_stats_collect(rc, bytesused);
}
diff --git a/include/media/v4l2-enc-cbr.h b/include/media/v4l2-enc-cbr.h
new file mode 100644
index 000000000000..6b66d121b0cc
--- /dev/null
+++ b/include/media/v4l2-enc-cbr.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * V4L2 stateless encoder constant bitrate rate control
+ *
+ * A codec independent frame-level rate control for the constant bitrate
+ * mode of the stateless encode cores.
+ */
+
+#ifndef _MEDIA_V4L2_ENC_CBR_H
+#define _MEDIA_V4L2_ENC_CBR_H
+
+#include <linux/types.h>
+#include <linux/videodev2.h>
+
+/**
+ * struct v4l2_enc_cbr - constant bitrate rate control state
+ * @qp_delta_intra: QP of the I pictures below the QP of the P pictures
+ * @bitrate: bitrate the state was set up for, 0 before the first picture
+ * @nominal: nominal picture size in bits (bitrate times frame interval)
+ * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
+ * @ratio: I/P picture size ratio in 1/16 units
+ * @gop: measured IDR period in pictures, 0 until two I pictures were seen
+ * @since_i: pictures coded since the last I picture
+ * @seen_i: an I picture was coded
+ * @intra: the current picture is an I picture
+ * @p_bits: running mean of the P picture sizes
+ * @i_bits: size of the last I picture
+ * @buffer_bits: accumulated difference between the coded and nominal sizes
+ * @plan_bits: planned size of the current picture
+ * @target_bits: size target of the current picture
+ *
+ * Only @qp_delta_intra is set by the user, after v4l2_enc_cbr_reset().
+ * @target_bits is valid from v4l2_enc_cbr_begin() of a picture until its
+ * v4l2_enc_cbr_end().
+ */
+struct v4l2_enc_cbr {
+ int qp_delta_intra;
+
+ u32 bitrate;
+ u64 nominal;
+ int qp_q8;
+ unsigned int ratio;
+ unsigned int gop;
+ unsigned int since_i;
+ bool seen_i;
+ bool intra;
+ u64 p_bits;
+ u64 i_bits;
+ s64 buffer_bits;
+ u64 plan_bits;
+ u64 target_bits;
+};
+
+void v4l2_enc_cbr_reset(struct v4l2_enc_cbr *cbr, int qp_delta_intra);
+int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
+ const struct v4l2_fract *timeperframe,
+ unsigned int samples, bool intra, int qp_min,
+ int qp_max);
+int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr);
+void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
+ const struct v4l2_fract *timeperframe, int qp_min,
+ int qp_max);
+
+#endif
diff --git a/include/media/v4l2-h264-enc-rc.h b/include/media/v4l2-h264-enc-rc.h
index 72d78a594e40..76781fe2d844 100644
--- a/include/media/v4l2-h264-enc-rc.h
+++ b/include/media/v4l2-h264-enc-rc.h
@@ -10,6 +10,7 @@
#include <linux/types.h>
#include <linux/v4l2-controls.h>
+#include <media/v4l2-enc-cbr.h>
#include <media/v4l2-h264.h>
#define V4L2_H264_ENC_RC_STATS_TYPE_COUNT 10
@@ -83,38 +84,6 @@ struct v4l2_h264_enc_rc_stats {
unsigned int index;
};
-/**
- * struct v4l2_h264_enc_rc_cbr - constant bitrate mode state
- * @bitrate: bitrate the state was set up for, 0 before the first picture
- * @nominal: nominal picture size in bits (bitrate times frame interval)
- * @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
- * @ratio: I/P picture size ratio in 1/16 units
- * @gop: measured IDR period in pictures, 0 until two I pictures were seen
- * @since_i: pictures coded since the last I picture
- * @seen_i: an I picture was coded
- * @intra: the current picture is an I picture
- * @p_bits: running mean of the P picture sizes
- * @i_bits: size of the last I picture
- * @buffer_bits: accumulated difference between the coded and nominal sizes
- * @plan_bits: planned size of the current picture
- * @target_bits: size target of the current picture
- */
-struct v4l2_h264_enc_rc_cbr {
- u32 bitrate;
- u64 nominal;
- int qp_q8;
- unsigned int ratio;
- unsigned int gop;
- unsigned int since_i;
- bool seen_i;
- bool intra;
- u64 p_bits;
- u64 i_bits;
- s64 buffer_bits;
- u64 plan_bits;
- u64 target_bits;
-};
-
struct v4l2_h264_enc_rc {
const struct v4l2_h264_enc_rc_ops *ops;
void *private_data;
@@ -124,7 +93,7 @@ struct v4l2_h264_enc_rc {
void *mode_data;
struct v4l2_h264_enc_rc_stats stats;
- struct v4l2_h264_enc_rc_cbr cbr;
+ struct v4l2_enc_cbr cbr;
unsigned long size;
unsigned int qp;
};
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 06/11] media: v4l2-core: Bound the I picture size in the encode CBR rate control
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
` (4 preceding siblings ...)
2026-11-10 10:05 ` [PATCH RFC 05/11] media: v4l2-core: Move the H.264 encode CBR rate control to a helper Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 07/11] media: v4l2-core: Halve the I picture QP increase of the encode CBR Arun Iyengar via B4 Relay
` (4 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The constant bitrate rate control codes I pictures a fixed QP delta below
the current QP state of the P pictures, whatever their size target. On
content whose P pictures get very cheap (static scenes), the state drifts
down over a GOP and the next IDR picture comes out far above its plan:
up to 33 times the nominal picture size at 1080p60 and 2 Mbit/s with
HEVC, 2.6 times its target and a third over its share of the GOP at the
largest I/P size ratio the plan allows. The P pictures of the GOP pay for
it in the end, but the bitrate over one second rises to 1.5 times the
target, and IDR sizes swing by up to 3 times from one GOP to the next.
Do what the Rockchip MPP library does for both codecs:
- code an I picture at the mean of the QP state now and at the previous
I picture (weighing the current state 70% when it rose from 33 or below
to 35 or above), so that a drift of the state over a GOP moves the next
I picture by half as much;
- after an I picture larger than its share of the GOP at the largest I/P
ratio (30), raise the QP of the following I pictures by up to 4 QP per
picture, in all at most a quarter QP per QP step below 45, and take the
increase back as I pictures come out below that share.
All-intra streams are unchanged: their QP state follows the I pictures
directly.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-enc-cbr.c | 98 +++++++++++++++++++++++++++++-----
include/media/v4l2-enc-cbr.h | 8 +++
2 files changed, 93 insertions(+), 13 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-enc-cbr.c b/drivers/media/v4l2-core/v4l2-enc-cbr.c
index 5bd10d494054..32f98052e981 100644
--- a/drivers/media/v4l2-core/v4l2-enc-cbr.c
+++ b/drivers/media/v4l2-core/v4l2-enc-cbr.c
@@ -4,7 +4,12 @@
*
* One QP state is kept for the stream: it is the QP of the P (and B)
* pictures, and I pictures are coded a fixed number of steps below it, so
- * that the quality of the reference pictures stays even across GOPs. Each
+ * that the quality of the reference pictures stays even across GOPs. The
+ * I pictures take the mean of the state now and at the previous I picture,
+ * so that a drift of the state over a GOP (static content, where P pictures
+ * are cheap) moves the size of the next I picture by half as much, and an
+ * I picture that exceeded its share of the GOP at the largest I/P size
+ * ratio raises the QP of the following I pictures until they fit. Each
* picture gets a size target from a GOP plan, in which an I picture is
* "ratio" times the size of a P picture, with the ratio and the IDR period
* measured on the stream. The difference between the coded size so far and
@@ -39,6 +44,15 @@
#define CBR_RATIO_INIT 160
#define CBR_RATIO_MIN 24
#define CBR_RATIO_MAX 480
+/*
+ * The I QP increase after oversized I pictures: at most 4 QP (1/256 units)
+ * per I picture for an I picture twice its largest share, and in all at most
+ * a quarter QP per QP step below 45, up to 6 QP (more where the QP is low,
+ * where a QP step changes the size by fewer bits).
+ */
+#define CBR_INTRA_ADD_STEP (4 * 256)
+#define CBR_INTRA_ADD_QP_END 45
+#define CBR_INTRA_ADD_MAX (6 * 256)
/* P pictures after an I picture when the I/P size ratio is measured */
#define CBR_RATIO_PICS 4
/* QP errors up to this are corrected by 1/8 per picture, 1/256 units */
@@ -85,9 +99,9 @@ static u64 cbr_nominal(u32 bitrate, const struct v4l2_fract *timeperframe)
timeperframe->denominator), 1);
}
-/* Planned size of an I or P picture of the current GOP */
-static u64 cbr_plan(const struct v4l2_enc_cbr *cbr, unsigned int fps,
- bool intra)
+/* Planned size of an I or P picture of the current GOP at an I/P ratio */
+static u64 cbr_plan_ratio(const struct v4l2_enc_cbr *cbr, unsigned int fps,
+ bool intra, unsigned int ratio)
{
unsigned int gop = cbr->gop;
u64 p;
@@ -98,13 +112,43 @@ static u64 cbr_plan(const struct v4l2_enc_cbr *cbr, unsigned int fps,
if (gop <= 1)
return cbr->nominal;
- p = div_u64(cbr->nominal * gop * 16, cbr->ratio + 16 * (gop - 1));
+ p = div_u64(cbr->nominal * gop * 16, ratio + 16 * (gop - 1));
if (intra)
- p = div_u64(p * cbr->ratio, 16);
+ p = div_u64(p * ratio, 16);
return max_t(u64, p, 1);
}
+static u64 cbr_plan(const struct v4l2_enc_cbr *cbr, unsigned int fps,
+ bool intra)
+{
+ return cbr_plan_ratio(cbr, fps, intra, cbr->ratio);
+}
+
+/*
+ * QP of an I picture: the mean of the QP state now and at the previous I
+ * picture (the current state weighs 70% when it rose from 33 or below to 35
+ * or above: content that got much harder), plus the increase after
+ * oversized I pictures, less the I QP delta. All-intra streams use the
+ * state alone, which then follows the I pictures fully.
+ */
+static int cbr_qp_intra(const struct v4l2_enc_cbr *cbr)
+{
+ int cur = cbr->qp_q8, ref = cbr->qp_intra_ref_q8, q8;
+
+ if (cbr->gop == 1 || ref < 0)
+ q8 = cur;
+ else if (cur >= 35 * 256 && ref <= 33 * 256)
+ q8 = (3 * ref + 7 * cur) / 10;
+ else
+ q8 = (ref + cur) / 2;
+
+ if (cbr->gop != 1)
+ q8 += cbr->qp_intra_add_q8;
+
+ return DIV_ROUND_CLOSEST(q8, 256) - cbr->qp_delta_intra;
+}
+
/*
* Start QP of an I picture from its size in bits per luma sample:
* 20.3 - 6 * log2(bits per sample), clipped to 15..45. This is the relation
@@ -165,6 +209,7 @@ int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
if (!cbr->bitrate) {
cbr->ratio = CBR_RATIO_INIT;
cbr->qp_q8 = -1;
+ cbr->qp_intra_ref_q8 = -1;
}
if (cbr->bitrate != bitrate || cbr->nominal != nominal) {
@@ -190,6 +235,9 @@ int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
cbr->qp_q8 = clamp_t(int, qp, qp_min, qp_max) * 256;
}
+ cbr->qp_pic = intra ? cbr_qp_intra(cbr) :
+ DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
+
/*
* The buffer holds the coded minus the nominal sizes. Within a GOP the
* plan expects it to go up by the excess of the I picture and back
@@ -214,6 +262,9 @@ int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
cbr->nominal, cbr->plan_bits, cbr->target_bits);
pr_debug(" buffer: %lld bits, expected: %lld bits, gop: %u, ratio: %u",
cbr->buffer_bits, expected, cbr->gop, cbr->ratio);
+ pr_debug(" intra: %d, picture QP %d, I state ref %d/256, I add %d/256",
+ intra, cbr->qp_pic, cbr->qp_intra_ref_q8,
+ cbr->qp_intra_add_q8);
return 0;
}
@@ -227,12 +278,7 @@ EXPORT_SYMBOL_GPL(v4l2_enc_cbr_begin);
*/
int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr)
{
- int qp = DIV_ROUND_CLOSEST(cbr->qp_q8, 256);
-
- if (cbr->intra)
- qp -= cbr->qp_delta_intra;
-
- return qp;
+ return cbr->qp_pic;
}
EXPORT_SYMBOL_GPL(v4l2_enc_cbr_qp);
@@ -248,9 +294,11 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
const struct v4l2_fract *timeperframe, int qp_min,
int qp_max)
{
- s64 limit = cbr->nominal * CBR_BUFFER_SECONDS * cbr_fps(timeperframe);
+ unsigned int fps = cbr_fps(timeperframe);
+ s64 limit = cbr->nominal * CBR_BUFFER_SECONDS * fps;
unsigned int ratio;
int err, step;
+ s64 cap, add;
if (!cbr->bitrate || !cbr->target_bits)
return;
@@ -285,6 +333,30 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
* estimate: only all-intra streams steer the QP fully with it.
*/
cbr->i_bits = bits;
+
+ /*
+ * Above its share of the GOP at the largest I/P ratio, an I
+ * picture is more than the P pictures of the plan can make
+ * up for: raise the QP of the next I pictures by up to 4 QP
+ * per picture (at twice the share), and lower the increase
+ * again as I pictures come out below the share.
+ */
+ cap = cbr_plan_ratio(cbr, fps, true, CBR_RATIO_MAX);
+ if (cbr->gop != 1 && (bits > cap || cbr->qp_intra_add_q8)) {
+ add = div64_s64(((s64)bits - cap) * CBR_INTRA_ADD_STEP,
+ cap);
+ add = clamp_t(s64, add, -CBR_INTRA_ADD_STEP,
+ CBR_INTRA_ADD_STEP) +
+ cbr->qp_intra_add_q8;
+ cbr->qp_intra_add_q8 =
+ clamp_t(s64, add, 0,
+ clamp((CBR_INTRA_ADD_QP_END -
+ cbr->qp_pic) * 64, 0,
+ CBR_INTRA_ADD_MAX));
+ }
+
+ /* The state the next I picture takes the mean with */
+ cbr->qp_intra_ref_q8 = cbr->qp_q8;
cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
}
diff --git a/include/media/v4l2-enc-cbr.h b/include/media/v4l2-enc-cbr.h
index 6b66d121b0cc..c47e459a1d1e 100644
--- a/include/media/v4l2-enc-cbr.h
+++ b/include/media/v4l2-enc-cbr.h
@@ -18,6 +18,11 @@
* @bitrate: bitrate the state was set up for, 0 before the first picture
* @nominal: nominal picture size in bits (bitrate times frame interval)
* @qp_q8: QP of the P pictures in 1/256 units, -1 before the first picture
+ * @qp_intra_ref_q8: @qp_q8 when the previous I picture was coded, -1 before
+ * the first I picture
+ * @qp_intra_add_q8: QP increase of the I pictures after I pictures larger
+ * than their share of the GOP at the largest I/P ratio
+ * @qp_pic: QP of the current picture
* @ratio: I/P picture size ratio in 1/16 units
* @gop: measured IDR period in pictures, 0 until two I pictures were seen
* @since_i: pictures coded since the last I picture
@@ -39,6 +44,9 @@ struct v4l2_enc_cbr {
u32 bitrate;
u64 nominal;
int qp_q8;
+ int qp_intra_ref_q8;
+ int qp_intra_add_q8;
+ int qp_pic;
unsigned int ratio;
unsigned int gop;
unsigned int since_i;
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 07/11] media: v4l2-core: Halve the I picture QP increase of the encode CBR
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
` (5 preceding siblings ...)
2026-11-10 10:05 ` [PATCH RFC 06/11] media: v4l2-core: Bound the I picture size in the encode CBR rate control Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 08/11] media: v4l2-core: Re-encode oversized pictures in " Arun Iyengar via B4 Relay
` (3 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
After an I picture above its share of the GOP at the largest I/P size
ratio, the constant bitrate rate control raises the QP of the following
I pictures. It adds the increase to the mean of the QP state now and at
the previous I picture, so all of it reaches the next I picture, and it
limits the increase by the QP of the I picture, four steps below the
state.
The Rockchip MPP library, whose behaviour this follows, adds the increase
to the current state before the mean, so that half of it reaches the I
picture, and limits it by the QP state. Do the same: an I picture that
came out large because of a passing scene is not penalised twice.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-enc-cbr.c | 36 ++++++++++++++++++----------------
1 file changed, 19 insertions(+), 17 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-enc-cbr.c b/drivers/media/v4l2-core/v4l2-enc-cbr.c
index 32f98052e981..70810b3fea23 100644
--- a/drivers/media/v4l2-core/v4l2-enc-cbr.c
+++ b/drivers/media/v4l2-core/v4l2-enc-cbr.c
@@ -47,8 +47,10 @@
/*
* The I QP increase after oversized I pictures: at most 4 QP (1/256 units)
* per I picture for an I picture twice its largest share, and in all at most
- * a quarter QP per QP step below 45, up to 6 QP (more where the QP is low,
- * where a QP step changes the size by fewer bits).
+ * a quarter QP per step of the QP state below 45, up to 6 QP (more where the
+ * QP is low, where a QP step changes the size by fewer bits). It raises the
+ * current state in the mean of cbr_qp_intra(), so half of it reaches the I
+ * pictures.
*/
#define CBR_INTRA_ADD_STEP (4 * 256)
#define CBR_INTRA_ADD_QP_END 45
@@ -126,25 +128,25 @@ static u64 cbr_plan(const struct v4l2_enc_cbr *cbr, unsigned int fps,
}
/*
- * QP of an I picture: the mean of the QP state now and at the previous I
- * picture (the current state weighs 70% when it rose from 33 or below to 35
- * or above: content that got much harder), plus the increase after
- * oversized I pictures, less the I QP delta. All-intra streams use the
- * state alone, which then follows the I pictures fully.
+ * QP of an I picture: the mean of the QP state now, raised by the increase
+ * after oversized I pictures, and the state at the previous I picture (the
+ * current state weighs 70% when it rose from 33 or below to 35 or above:
+ * content that got much harder), less the I QP delta. All-intra streams use
+ * the state alone, which then follows the I pictures fully.
*/
static int cbr_qp_intra(const struct v4l2_enc_cbr *cbr)
{
int cur = cbr->qp_q8, ref = cbr->qp_intra_ref_q8, q8;
- if (cbr->gop == 1 || ref < 0)
+ if (cbr->gop == 1 || ref < 0) {
q8 = cur;
- else if (cur >= 35 * 256 && ref <= 33 * 256)
- q8 = (3 * ref + 7 * cur) / 10;
- else
- q8 = (ref + cur) / 2;
-
- if (cbr->gop != 1)
- q8 += cbr->qp_intra_add_q8;
+ } else {
+ cur += cbr->qp_intra_add_q8;
+ if (cur >= 35 * 256 && ref <= 33 * 256)
+ q8 = (3 * ref + 7 * cur) / 10;
+ else
+ q8 = (ref + cur) / 2;
+ }
return DIV_ROUND_CLOSEST(q8, 256) - cbr->qp_delta_intra;
}
@@ -350,8 +352,8 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
cbr->qp_intra_add_q8;
cbr->qp_intra_add_q8 =
clamp_t(s64, add, 0,
- clamp((CBR_INTRA_ADD_QP_END -
- cbr->qp_pic) * 64, 0,
+ clamp(CBR_INTRA_ADD_QP_END * 64 -
+ cbr->qp_q8 / 4, 0,
CBR_INTRA_ADD_MAX));
}
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 08/11] media: v4l2-core: Re-encode oversized pictures in the encode CBR
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
` (6 preceding siblings ...)
2026-11-10 10:05 ` [PATCH RFC 07/11] media: v4l2-core: Halve the I picture QP increase of the encode CBR Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 09/11] media: v4l2-core: Fix all-intra streams in the encode CBR rate control Arun Iyengar via B4 Relay
` (2 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
A scene change in a P picture can make it many times larger than its size
target: the QP state of the constant bitrate rate control was set for the
content before. The rate control repays the excess over the next pictures,
but the peak bitrate around that picture, together with an I picture
nearby, can exceed what a decoder buffer of the stream takes.
The Rockchip MPP library codes such a picture a second time with a higher
QP. Do the same in v4l2-enc-cbr, for drivers that can encode a picture
again: v4l2_enc_cbr_reencode() asks for a second pass when the picture is
above 1.5 times its target (I pictures) or 3 times (P pictures), when it
exceeds its target by more than 0.15 s of the bitrate (it alone moves the
bitrate of a three second window by more than 5%), and when that window
is more than 10% off the target. The second pass is coded 1, 2 or 3 QP
higher for a picture above, 1.5 times or twice that threshold. A picture
is coded at most twice. The window is a running sum of the picture sizes
that decays over three seconds, which needs no history buffer.
The QP state then learns from the size of the second pass, with its error
counted at the QP of the state.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-enc-cbr.c | 94 +++++++++++++++++++++++++++++++++-
include/media/v4l2-enc-cbr.h | 9 ++++
2 files changed, 101 insertions(+), 2 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-enc-cbr.c b/drivers/media/v4l2-core/v4l2-enc-cbr.c
index 70810b3fea23..28826e32c418 100644
--- a/drivers/media/v4l2-core/v4l2-enc-cbr.c
+++ b/drivers/media/v4l2-core/v4l2-enc-cbr.c
@@ -16,6 +16,11 @@
* the size the plan expects at this point of the GOP is spread over the next
* pictures, so that the bitrate converges to the target.
*
+ * A picture far above its size target that also moves the bitrate of the
+ * last three seconds well away from the target can be coded a second time
+ * with a higher QP (the re-encode of the Rockchip MPP library): I pictures
+ * above 1.5 times their target, P pictures above 3 times (a scene change).
+ *
* After each picture the QP state moves by a fraction of the QP error
* 6 * log2(size / target) (a QP step of 6 halves the size): slowly for small
* errors, so that the QP does not follow the size noise from picture to
@@ -29,6 +34,7 @@
#include <linux/export.h>
#include <linux/int_log.h>
+#include <linux/math.h>
#include <linux/math64.h>
#include <linux/minmax.h>
#include <linux/module.h>
@@ -66,6 +72,16 @@
*/
#define CBR_STEP_UP_MAX (4 * 256)
#define CBR_STEP_DOWN_MAX (2 * 256)
+/*
+ * Re-encode: size thresholds in halves of the target (I 1.5x, P 3x), the
+ * least excess over the target (1/20 of the bits of the three second
+ * window, i.e. 0.15 s of the bitrate) and the least deviation of the window
+ * from the target (1/10).
+ */
+#define CBR_REENC_THR_INTRA 3
+#define CBR_REENC_THR_INTER 6
+#define CBR_REENC_JUMP_DIV 20
+#define CBR_REENC_DEV_DIV 10
/* log2(x) in 8.8 fixed point */
static int cbr_log2_q8(u64 x)
@@ -218,6 +234,7 @@ int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
cbr->bitrate = bitrate;
cbr->nominal = nominal;
cbr->buffer_bits = 0;
+ cbr->window_bits = nominal * CBR_BUFFER_SECONDS * fps;
}
if (intra) {
@@ -258,6 +275,7 @@ int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
max_t(u64, cbr->plan_bits / 4, 8),
cbr->plan_bits * 2);
cbr->intra = intra;
+ cbr->qp_reenc = 0;
pr_debug("+ v4l2-enc-cbr: begin");
pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
@@ -284,6 +302,70 @@ int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr)
}
EXPORT_SYMBOL_GPL(v4l2_enc_cbr_qp);
+/**
+ * v4l2_enc_cbr_reencode() - decide on coding the picture a second time
+ * @cbr: the rate control state
+ * @bits: size of the coded picture in bits
+ * @timeperframe: frame interval
+ * @qp: QP the picture was coded with, within the QP limits of the codec
+ * @qp_max: highest QP of the picture
+ *
+ * Called after a picture was coded and before v4l2_enc_cbr_end(), by
+ * drivers that can code the same picture again. A picture is coded at most
+ * twice. When it returns true, v4l2_enc_cbr_qp() gives the higher QP of the
+ * second pass, which keeps the size target; v4l2_enc_cbr_end() is then
+ * called with the size of the second pass only.
+ *
+ * Return: true if the picture should be coded again.
+ */
+bool v4l2_enc_cbr_reencode(struct v4l2_enc_cbr *cbr, u64 bits,
+ const struct v4l2_fract *timeperframe, int qp,
+ int qp_max)
+{
+ unsigned int fps = cbr_fps(timeperframe);
+ u64 n = CBR_BUFFER_SECONDS * fps;
+ u64 window = cbr->nominal * n;
+ u64 thr, ins;
+ int add;
+
+ if (!cbr->bitrate || !cbr->target_bits || cbr->qp_reenc)
+ return false;
+
+ thr = div_u64(cbr->target_bits * (cbr->intra ? CBR_REENC_THR_INTRA :
+ CBR_REENC_THR_INTER), 2);
+ if (bits <= thr)
+ return false;
+
+ /* The picture alone moves the window by more than 5% ... */
+ if (bits - cbr->target_bits <= div_u64(window, CBR_REENC_JUMP_DIV))
+ return false;
+
+ /* ... and the window is more than 10% off the target. */
+ ins = cbr->window_bits - div64_u64(cbr->window_bits, n) + bits;
+ if (abs_diff(ins, window) <= div_u64(window, CBR_REENC_DEV_DIV))
+ return false;
+
+ if (bits > 2 * thr)
+ add = 3;
+ else if (2 * bits > 3 * thr)
+ add = 2;
+ else
+ add = 1;
+
+ if (qp + add > qp_max)
+ return false;
+
+ cbr->qp_pic = qp + add;
+ cbr->qp_reenc = add;
+
+ pr_debug("+ v4l2-enc-cbr: re-encode");
+ pr_debug(" size: %llu bits, target: %llu bits, window: %llu bits, picture QP %d",
+ bits, cbr->target_bits, ins, cbr->qp_pic);
+
+ return true;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_reencode);
+
/**
* v4l2_enc_cbr_end() - account for a coded picture
* @cbr: the rate control state
@@ -307,9 +389,17 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
cbr->buffer_bits = clamp_t(s64, cbr->buffer_bits + (s64)bits -
(s64)cbr->nominal, -limit, limit);
+ /* Sum of the picture sizes over about three seconds */
+ cbr->window_bits = cbr->window_bits + bits -
+ div64_u64(cbr->window_bits,
+ CBR_BUFFER_SECONDS * fps);
- /* QP error of the picture, 1/256 units */
- err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits));
+ /*
+ * QP error of the picture, 1/256 units. A re-encoded picture was coded
+ * above the QP of the state: its error at the state counts that too.
+ */
+ err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits)) +
+ cbr->qp_reenc * 256;
step = err / 8;
if (err > CBR_ERR_SLOW)
diff --git a/include/media/v4l2-enc-cbr.h b/include/media/v4l2-enc-cbr.h
index c47e459a1d1e..867e176ecb36 100644
--- a/include/media/v4l2-enc-cbr.h
+++ b/include/media/v4l2-enc-cbr.h
@@ -23,6 +23,8 @@
* @qp_intra_add_q8: QP increase of the I pictures after I pictures larger
* than their share of the GOP at the largest I/P ratio
* @qp_pic: QP of the current picture
+ * @qp_reenc: QP increase of the second pass of the current picture, 0 if
+ * it was coded once
* @ratio: I/P picture size ratio in 1/16 units
* @gop: measured IDR period in pictures, 0 until two I pictures were seen
* @since_i: pictures coded since the last I picture
@@ -31,6 +33,8 @@
* @p_bits: running mean of the P picture sizes
* @i_bits: size of the last I picture
* @buffer_bits: accumulated difference between the coded and nominal sizes
+ * @window_bits: sum of the coded sizes over about three seconds (each size
+ * decays by 1/(3 * fps) per picture)
* @plan_bits: planned size of the current picture
* @target_bits: size target of the current picture
*
@@ -47,6 +51,7 @@ struct v4l2_enc_cbr {
int qp_intra_ref_q8;
int qp_intra_add_q8;
int qp_pic;
+ int qp_reenc;
unsigned int ratio;
unsigned int gop;
unsigned int since_i;
@@ -55,6 +60,7 @@ struct v4l2_enc_cbr {
u64 p_bits;
u64 i_bits;
s64 buffer_bits;
+ u64 window_bits;
u64 plan_bits;
u64 target_bits;
};
@@ -65,6 +71,9 @@ int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
unsigned int samples, bool intra, int qp_min,
int qp_max);
int v4l2_enc_cbr_qp(const struct v4l2_enc_cbr *cbr);
+bool v4l2_enc_cbr_reencode(struct v4l2_enc_cbr *cbr, u64 bits,
+ const struct v4l2_fract *timeperframe, int qp,
+ int qp_max);
void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
const struct v4l2_fract *timeperframe, int qp_min,
int qp_max);
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 09/11] media: v4l2-core: Fix all-intra streams in the encode CBR rate control
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
` (7 preceding siblings ...)
2026-11-10 10:05 ` [PATCH RFC 08/11] media: v4l2-core: Re-encode oversized pictures in " Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 10/11] media: v4l2-core: Add a peak bitrate to the encode " Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 11/11] media: h264: Add a VBR mode to the stateless " Arun Iyengar via B4 Relay
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
All-intra streams (an IDR period of 1) have two problems in the constant
bitrate rate control:
- the QP state, which is the QP of the P pictures, is limited to the QP
range, and the I pictures are coded the I QP delta below it, so they
never reach the highest QP: at targets that need it the bitrate stays
far above the target (2.3 times at 1080p30 and 1 Mbit/s with QP 49
instead of 51);
- the start QP is estimated for an I picture of a GOP of two seconds,
which gets about nine times the nominal size, so the first pictures of
an all-intra stream come out several times their size until the state
has climbed: the largest one second window of a 1080p30 stream at
4 Mbit/s is 1.55 times the bitrate.
Shift the range of the state by the I QP delta in all-intra streams, and
when the second picture is an I picture again, set the state from the
size of the first one at its QP. In a host simulation against a model
encoder (sizes halving every 6 QP, 720p and 1080p, 256 kbit/s to
20 Mbit/s), the all-intra bitrate is then within 0.1% of the target
wherever QP 51 can reach it, and the largest one second window drops to
1.02-1.11 times the bitrate (was up to 1.88). Streams with P pictures are
not changed.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-enc-cbr.c | 27 ++++++++++++++++++++++++---
1 file changed, 24 insertions(+), 3 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-enc-cbr.c b/drivers/media/v4l2-core/v4l2-enc-cbr.c
index 28826e32c418..87db60469182 100644
--- a/drivers/media/v4l2-core/v4l2-enc-cbr.c
+++ b/drivers/media/v4l2-core/v4l2-enc-cbr.c
@@ -239,8 +239,22 @@ int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
if (intra) {
/* An I picture closes the GOP: its length is the IDR period. */
- if (cbr->seen_i)
+ if (cbr->seen_i) {
+ /*
+ * The second picture of an all-intra stream: the size
+ * of the first one at its QP tells the QP of the
+ * nominal size better than the start estimate, which
+ * assumed a GOP with P pictures.
+ */
+ if (!cbr->gop && !cbr->since_i)
+ cbr->qp_q8 = clamp_t(int, (cbr->qp_pic +
+ cbr->qp_delta_intra) * 256 +
+ 6 * (cbr_log2_q8(cbr->i_bits) -
+ cbr_log2_q8(nominal)),
+ (qp_min + cbr->qp_delta_intra) * 256,
+ (qp_max + cbr->qp_delta_intra) * 256);
cbr->gop = cbr->since_i + 1;
+ }
cbr->since_i = 0;
cbr->seen_i = true;
}
@@ -381,7 +395,7 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
unsigned int fps = cbr_fps(timeperframe);
s64 limit = cbr->nominal * CBR_BUFFER_SECONDS * fps;
unsigned int ratio;
- int err, step;
+ int err, step, shift;
s64 cap, add;
if (!cbr->bitrate || !cbr->target_bits)
@@ -452,7 +466,14 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
cbr->qp_q8 += cbr->gop == 1 ? step : step / 4;
}
- cbr->qp_q8 = clamp_t(int, cbr->qp_q8, qp_min * 256, qp_max * 256);
+ /*
+ * The state is the QP of the P pictures. All-intra streams code their
+ * pictures at the state less the I QP delta, so they need the QP range
+ * shifted up by it to reach the highest QP.
+ */
+ shift = cbr->gop == 1 ? cbr->qp_delta_intra : 0;
+ cbr->qp_q8 = clamp_t(int, cbr->qp_q8, (qp_min + shift) * 256,
+ (qp_max + shift) * 256);
pr_debug("+ v4l2-enc-cbr: end");
pr_debug(" size: %llu bits, target: %llu bits, qp error: %d/256, qp: %d/256",
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 10/11] media: v4l2-core: Add a peak bitrate to the encode rate control
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
` (8 preceding siblings ...)
2026-11-10 10:05 ` [PATCH RFC 09/11] media: v4l2-core: Fix all-intra streams in the encode CBR rate control Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
2026-11-10 10:05 ` [PATCH RFC 11/11] media: h264: Add a VBR mode to the stateless " Arun Iyengar via B4 Relay
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The stateless encode cores read V4L2_CID_MPEG_VIDEO_BITRATE_PEAK for
V4L2_MPEG_VIDEO_BITRATE_MODE_VBR, but their rate control only has a
constant bitrate model.
Give the model a peak bitrate, which makes it a variable bitrate one: the
bitrate is met on average over the stream, and the peak bitrate bounds
every second of it.
- The deviation of the coded sizes from the GOP plan is spread over two
seconds of pictures instead of 8 pictures, so that the picture sizes
can follow the content for a while.
- The size target of a picture is limited to what is left of the peak in
the last second (the sizes of up to 128 pictures are kept for it), and
its QP is raised by the QP steps between its expected size and that: a
P picture is expected at the size of the last P picture, an I picture
at the larger of the last I picture and the last P picture times the
measured I/P size ratio, each at the QP difference. The QP state counts
the raise as part of the QP error of the picture.
A picture the history cannot foresee (an I picture at a cut to much
harder content) can still take the second over the peak; re-encoding is
up to the driver.
In a host simulation against a model encoder (sizes halving every 6 QP;
720p and 1080p, 256 kbit/s to 20 Mbit/s, peak 1.5 times the bitrate, IDR
periods 1, 10, 30 and 120, a 3x and 0.33x change of the content), the
bitrate is within 0.3% of the target and every second within the peak
wherever QP 51 can reach the target. With the content alternating
between two complexities 4x apart every 3 seconds, the largest second is
0.79-1.0 times the peak, or 1.24-1.26 times where the cut falls on an I
picture. The constant bitrate mode is not changed.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-enc-cbr.c | 116 +++++++++++++++++++++++++++++++--
include/media/v4l2-enc-cbr.h | 24 ++++++-
2 files changed, 135 insertions(+), 5 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-enc-cbr.c b/drivers/media/v4l2-core/v4l2-enc-cbr.c
index 87db60469182..c9e806bbabcb 100644
--- a/drivers/media/v4l2-core/v4l2-enc-cbr.c
+++ b/drivers/media/v4l2-core/v4l2-enc-cbr.c
@@ -30,6 +30,13 @@
* The QP to size relation (6 QP steps per halving) is the same for H.264
* and HEVC, so the model does not depend on the codec. Only the QP delta of
* the I pictures is a parameter.
+ *
+ * With a peak bitrate (v4l2_enc_cbr_set_peak()), the same model gives a
+ * variable bitrate: the bitrate is the average over the stream and the
+ * peak bitrate a bound over any second of it. The deviation from the plan
+ * is spread over two seconds instead of a few pictures, so that the size of
+ * the pictures can follow the content, and a picture that would take the
+ * last second over the peak gets a lower size target and a higher QP.
*/
#include <linux/export.h>
@@ -44,6 +51,8 @@
/* The accumulated size error is spread over this many pictures. */
#define CBR_WINDOW 8
+/* With a peak bitrate, over this many seconds of pictures */
+#define VBR_WINDOW_SECONDS 2
/* Limit of the accumulated size error, in seconds of the bitrate */
#define CBR_BUFFER_SECONDS 3
/* I/P size ratio, 1/16 units: initial value (10x) and limits (1.5x .. 30x) */
@@ -181,6 +190,76 @@ static int cbr_qp_initial(u64 bits, unsigned int samples)
return clamp(DIV_ROUND_CLOSEST(qp_q8, 256), 15, 45);
}
+/* Sum of the sizes of the last @count pictures */
+static u64 vbr_window_sum(const struct v4l2_enc_cbr *cbr, unsigned int count)
+{
+ unsigned int i, index = cbr->peak_index;
+ u64 sum = 0;
+
+ count = min(count, cbr->peak_count);
+ for (i = 0; i < count; i++) {
+ index = (index + V4L2_ENC_CBR_PEAK_PICS - 1) %
+ V4L2_ENC_CBR_PEAK_PICS;
+ sum += cbr->peak_bits[index];
+ }
+
+ return sum;
+}
+
+/*
+ * QP steps (1/256 units) that bring a picture of @bits at @qp, coded at
+ * @qp_pic instead, down to @room: 6 QP steps per halving.
+ */
+static int vbr_qp_add(u64 bits, int qp, int qp_pic, u64 room)
+{
+ if (!bits)
+ return 0;
+
+ return 6 * (cbr_log2_q8(bits) - cbr_log2_q8(room)) -
+ (qp_pic - qp) * 256;
+}
+
+/*
+ * Keep the last second (at most V4L2_ENC_CBR_PEAK_PICS pictures) within the
+ * peak bitrate: limit the size target of the picture to what is left of the
+ * peak, and raise its QP by the QP steps between its expected size and
+ * that. A P picture is expected at the size of the last one, an I picture
+ * at the larger of the last I picture and the last P picture times the I/P
+ * size ratio (which follows a change of the content sooner), each at the
+ * QP difference.
+ */
+static void vbr_bound(struct v4l2_enc_cbr *cbr,
+ const struct v4l2_fract *timeperframe, unsigned int fps,
+ int qp_max)
+{
+ unsigned int pics = clamp(fps, 1U, V4L2_ENC_CBR_PEAK_PICS);
+ u64 peak = cbr_nominal(max(cbr->bitrate_peak, cbr->bitrate),
+ timeperframe) * pics;
+ u64 used = vbr_window_sum(cbr, pics - 1);
+ u64 room = max_t(u64, peak > used ? peak - used : 0, 8);
+ int add;
+
+ cbr->target_bits = min(cbr->target_bits, room);
+
+ if (cbr->intra)
+ add = max(vbr_qp_add(cbr->i_bits, cbr->i_qp, cbr->qp_pic, room),
+ vbr_qp_add(div_u64(cbr->p_last_bits * cbr->ratio, 16),
+ cbr->p_qp - cbr->qp_delta_intra,
+ cbr->qp_pic, room));
+ else
+ add = vbr_qp_add(cbr->p_last_bits, cbr->p_qp, cbr->qp_pic,
+ room);
+
+ add = min(DIV_ROUND_UP(add, 256), qp_max - cbr->qp_pic);
+ if (add <= 0)
+ return;
+
+ cbr->qp_pic += add;
+ cbr->qp_peak_add = add;
+
+ pr_debug(" peak: %llu bits left of %llu, QP +%d", room, peak, add);
+}
+
/**
* v4l2_enc_cbr_reset() - reset the rate control state
* @cbr: the rate control state
@@ -195,6 +274,22 @@ void v4l2_enc_cbr_reset(struct v4l2_enc_cbr *cbr, int qp_delta_intra)
}
EXPORT_SYMBOL_GPL(v4l2_enc_cbr_reset);
+/**
+ * v4l2_enc_cbr_set_peak() - set the peak bitrate
+ * @cbr: the rate control state
+ * @bitrate_peak: peak bitrate in bits per second, 0 for a constant bitrate
+ *
+ * A non-zero peak bitrate makes the rate control a variable bitrate one:
+ * the bitrate of v4l2_enc_cbr_begin() is met on average, and no second of
+ * the stream goes over the peak bitrate (a peak below the bitrate counts as
+ * the bitrate). Takes effect from the next v4l2_enc_cbr_begin().
+ */
+void v4l2_enc_cbr_set_peak(struct v4l2_enc_cbr *cbr, u32 bitrate_peak)
+{
+ cbr->bitrate_peak = bitrate_peak;
+}
+EXPORT_SYMBOL_GPL(v4l2_enc_cbr_set_peak);
+
/**
* v4l2_enc_cbr_begin() - set up the size target of the next picture
* @cbr: the rate control state
@@ -284,12 +379,17 @@ int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
target = (s64)cbr->plan_bits -
div_s64(cbr->buffer_bits - max_t(s64, expected, 0),
- CBR_WINDOW);
+ cbr->bitrate_peak ? VBR_WINDOW_SECONDS * fps :
+ CBR_WINDOW);
cbr->target_bits = clamp_t(s64, target,
max_t(u64, cbr->plan_bits / 4, 8),
cbr->plan_bits * 2);
cbr->intra = intra;
cbr->qp_reenc = 0;
+ cbr->qp_peak_add = 0;
+
+ if (cbr->bitrate_peak)
+ vbr_bound(cbr, timeperframe, fps, qp_max);
pr_debug("+ v4l2-enc-cbr: begin");
pr_debug(" nominal: %llu bits, plan: %llu bits, target: %llu bits",
@@ -408,12 +508,17 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
div64_u64(cbr->window_bits,
CBR_BUFFER_SECONDS * fps);
+ cbr->peak_bits[cbr->peak_index] = min_t(u64, bits, U32_MAX);
+ cbr->peak_index = (cbr->peak_index + 1) % V4L2_ENC_CBR_PEAK_PICS;
+ cbr->peak_count = min(cbr->peak_count + 1, V4L2_ENC_CBR_PEAK_PICS);
+
/*
- * QP error of the picture, 1/256 units. A re-encoded picture was coded
- * above the QP of the state: its error at the state counts that too.
+ * QP error of the picture, 1/256 units. A picture coded above the QP
+ * of the state (re-encoded, or raised for the peak bitrate) counts
+ * that too: the state would have given a picture that much larger.
*/
err = 6 * (cbr_log2_q8(bits) - cbr_log2_q8(cbr->target_bits)) +
- cbr->qp_reenc * 256;
+ (cbr->qp_reenc + cbr->qp_peak_add) * 256;
step = err / 8;
if (err > CBR_ERR_SLOW)
@@ -425,6 +530,8 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
if (!cbr->intra) {
cbr->qp_q8 += step;
cbr->p_bits = cbr->p_bits ? (7 * cbr->p_bits + bits) / 8 : bits;
+ cbr->p_last_bits = bits;
+ cbr->p_qp = clamp(cbr->qp_pic, qp_min, qp_max);
cbr->since_i++;
/* I/P size ratio, once a few P pictures were coded */
@@ -439,6 +546,7 @@ void v4l2_enc_cbr_end(struct v4l2_enc_cbr *cbr, u64 bits,
* estimate: only all-intra streams steer the QP fully with it.
*/
cbr->i_bits = bits;
+ cbr->i_qp = clamp(cbr->qp_pic, qp_min, qp_max);
/*
* Above its share of the GOP at the largest I/P ratio, an I
diff --git a/include/media/v4l2-enc-cbr.h b/include/media/v4l2-enc-cbr.h
index 867e176ecb36..4f4874c41d2a 100644
--- a/include/media/v4l2-enc-cbr.h
+++ b/include/media/v4l2-enc-cbr.h
@@ -12,6 +12,9 @@
#include <linux/types.h>
#include <linux/videodev2.h>
+/* Pictures of the peak bitrate window: one second, up to 128 pictures. */
+#define V4L2_ENC_CBR_PEAK_PICS 128U
+
/**
* struct v4l2_enc_cbr - constant bitrate rate control state
* @qp_delta_intra: QP of the I pictures below the QP of the P pictures
@@ -37,8 +40,17 @@
* decays by 1/(3 * fps) per picture)
* @plan_bits: planned size of the current picture
* @target_bits: size target of the current picture
+ * @bitrate_peak: peak bitrate in bits per second, 0 for a constant bitrate
+ * @qp_peak_add: QP increase of the current picture for the peak bitrate
+ * @i_qp: QP of the last I picture
+ * @p_last_bits: size of the last P picture
+ * @p_qp: QP of the last P picture
+ * @peak_bits: sizes of the last pictures, for the peak bitrate
+ * @peak_index: next entry of @peak_bits
+ * @peak_count: valid entries of @peak_bits
*
- * Only @qp_delta_intra is set by the user, after v4l2_enc_cbr_reset().
+ * Only @qp_delta_intra is set by the user, after v4l2_enc_cbr_reset(), and
+ * @bitrate_peak by v4l2_enc_cbr_set_peak().
* @target_bits is valid from v4l2_enc_cbr_begin() of a picture until its
* v4l2_enc_cbr_end().
*/
@@ -63,9 +75,19 @@ struct v4l2_enc_cbr {
u64 window_bits;
u64 plan_bits;
u64 target_bits;
+
+ u32 bitrate_peak;
+ int qp_peak_add;
+ int i_qp;
+ u64 p_last_bits;
+ int p_qp;
+ u32 peak_bits[V4L2_ENC_CBR_PEAK_PICS];
+ unsigned int peak_index;
+ unsigned int peak_count;
};
void v4l2_enc_cbr_reset(struct v4l2_enc_cbr *cbr, int qp_delta_intra);
+void v4l2_enc_cbr_set_peak(struct v4l2_enc_cbr *cbr, u32 bitrate_peak);
int v4l2_enc_cbr_begin(struct v4l2_enc_cbr *cbr, u32 bitrate,
const struct v4l2_fract *timeperframe,
unsigned int samples, bool intra, int qp_min,
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH RFC 11/11] media: h264: Add a VBR mode to the stateless encode rate control
2026-11-10 10:05 [PATCH RFC 00/11] media: h264: Stateless encode core follow-ups: parameter sets, POC type 2, CBR and VBR Arun Iyengar via B4 Relay
` (9 preceding siblings ...)
2026-11-10 10:05 ` [PATCH RFC 10/11] media: v4l2-core: Add a peak bitrate to the encode " Arun Iyengar via B4 Relay
@ 2026-11-10 10:05 ` Arun Iyengar via B4 Relay
10 siblings, 0 replies; 12+ messages in thread
From: Arun Iyengar via B4 Relay @ 2026-11-10 10:05 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Paul Kocialkowski
Cc: linux-media, linux-kernel, Nicolas Dufresne, Hans Verkuil,
Marco Felsch, Michael Tretter, Arun Iyengar
From: Arun Iyengar <aiyengar1990@gmail.com>
The core reads V4L2_CID_MPEG_VIDEO_BITRATE and _BITRATE_PEAK for
V4L2_MPEG_VIDEO_BITRATE_MODE_VBR, but has no mode for it, so a driver
offering VBR fails the first picture with -EINVAL.
Add one with the encode rate control and its peak bitrate: the bitrate is
met on average and every second stays within the peak (a peak below the
bitrate counts as the bitrate). Drivers offer it with the VBR entry of
V4L2_CID_MPEG_VIDEO_BITRATE_MODE and V4L2_CID_MPEG_VIDEO_BITRATE_PEAK.
Signed-off-by: Arun Iyengar <aiyengar1990@gmail.com>
---
drivers/media/v4l2-core/v4l2-h264-enc-rc.c | 24 ++++++++++++++++++++++++
drivers/media/v4l2-core/v4l2-h264-enc.c | 3 ++-
2 files changed, 26 insertions(+), 1 deletion(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
index c4f1718ead0e..691c80c2a950 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc-rc.c
@@ -218,10 +218,34 @@ static const struct v4l2_h264_enc_rc_mode cbr_mode = {
.complete = cbr_complete,
};
+/*
+ * Variable bitrate mode: the same rate control with the peak bitrate, which
+ * meets the bitrate on average and keeps every second within the peak.
+ */
+static int vbr_measure(struct v4l2_h264_enc_rc *rc, unsigned long *size)
+{
+ struct v4l2_h264_enc_state *state = rc->state;
+
+ v4l2_enc_cbr_set_peak(&rc->cbr, max(state->bitrate_peak,
+ state->bitrate));
+
+ return cbr_measure(rc, size);
+}
+
+static const struct v4l2_h264_enc_rc_mode vbr_mode = {
+ .name = "variable bitrate",
+ .type = V4L2_H264_ENC_RC_TYPE_VBR,
+ .init = cbr_init,
+ .measure = vbr_measure,
+ .estimate = cbr_estimate,
+ .complete = cbr_complete,
+};
+
static const struct v4l2_h264_enc_rc_mode *modes[] = {
&direct_mode,
&cq_mode,
&cbr_mode,
+ &vbr_mode,
};
static int mode_prepare(struct v4l2_h264_enc_rc *rc)
diff --git a/drivers/media/v4l2-core/v4l2-h264-enc.c b/drivers/media/v4l2-core/v4l2-h264-enc.c
index 9745c0e85ee5..66473231df94 100644
--- a/drivers/media/v4l2-core/v4l2-h264-enc.c
+++ b/drivers/media/v4l2-core/v4l2-h264-enc.c
@@ -1273,7 +1273,8 @@ static int rc_step(struct v4l2_h264_enc *enc)
if (rc->mode) {
pr_debug(" rc mode: %s", rc->mode->name);
- if (rc->mode->type == V4L2_H264_ENC_RC_TYPE_CBR) {
+ if (rc->mode->type == V4L2_H264_ENC_RC_TYPE_CBR ||
+ rc->mode->type == V4L2_H264_ENC_RC_TYPE_VBR) {
pr_debug(" rc bitrate: %u bits/s", state->bitrate);
pr_debug(" rc target: %lu bytes", rc->size);
}
--
2.55.0
^ permalink raw reply [flat|nested] 12+ messages in thread