mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Neil Armstrong <neil.armstrong@linaro.org>
To: Vikash Garodia <vikash.garodia@oss.qualcomm.com>,
	 Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>,
	 Abhinav Kumar <abhinav.kumar@linux.dev>,
	Bryan O'Donoghue <bod@kernel.org>,
	 Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org,
	 linux-kernel@vger.kernel.org,
	Neil Armstrong <neil.armstrong@linaro.org>
Subject: [PATCH 2/2] media: qcom: iris: venc: drop operating rate to match spec
Date: Tue, 17 Nov 2026 16:21:24 +0100	[thread overview]
Message-ID: <20261117-topic-iris-encoder-fixes-v1-2-169ce990d12d@linaro.org> (raw)
In-Reply-To: <20261117-topic-iris-encoder-fixes-v1-0-169ce990d12d@linaro.org>

Per the V4L2 Stateful Video Encoder UAPI documentation:
  Set the raw frame interval on the OUTPUT queue via VIDIOC_S_PARM().
  This also sets the coded frame interval on the CAPTURE queue to the same value.

This implies setting the frame interval on the OUTPUT queue _should_ be
supported as a single operation, and setting the frame interval on the
CAPTURE queue can be considered as optional since the S_PARM on
the OUTPUT queue will set the same frame interval on the CAPTURE
queue.

The current iris implementation maps:
 - inst->operating_rate to OUTPUT
 - inst->frame_rate to CAPTURE

Only inst->frame_rate is set to the firmware, and inst->operating_rate
is used in combination with inst->frame_rate to calculate
the entire iris bandwidth in iris_check_core_mbps() plus
returing the frame interval with G_PARM on the CAPTURE queue.

So per the Documentation, the two major V4L2 Stateful Video Encoder
client implementation (FFMpeg & GStreamer) only calls S_PARM
on the OUTPUT queue, leaving inst->frame_rate to DEFAULT_FPS,
and only setting inst->operating_rate.

Since inst->operating_rate is currently useless and breaks the
conformance, let's just remove operating_rate entirely
and just set inst->frame_rate only with S_PARM on the OUTPUT
queue.

The two rate variables were probably added to allow scaling
the iris OPP to allow faster encode speed, but wasn't yet
properly implemented and breaks the S_PARM compliance.

Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org>
---
 drivers/media/platform/qcom/iris/iris_instance.h   |  2 -
 .../platform/qcom/iris/iris_platform_common.h      |  1 -
 .../media/platform/qcom/iris/iris_platform_milos.h |  1 -
 .../platform/qcom/iris/iris_platform_qcs8300.h     |  1 -
 .../platform/qcom/iris/iris_platform_sm8550.h      |  1 -
 .../media/platform/qcom/iris/iris_platform_vpu2.c  |  1 -
 .../platform/qcom/iris/iris_platform_vpu_ar50lt.c  |  1 -
 drivers/media/platform/qcom/iris/iris_utils.c      |  8 ++--
 drivers/media/platform/qcom/iris/iris_venc.c       | 53 ++++++----------------
 9 files changed, 18 insertions(+), 51 deletions(-)

diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
index 771269e705d9..8eaeae5aff5f 100644
--- a/drivers/media/platform/qcom/iris/iris_instance.h
+++ b/drivers/media/platform/qcom/iris/iris_instance.h
@@ -59,7 +59,6 @@ struct iris_hfi_session_ops;
  * @last_buf_ns: start time of received input buffer for current one second FPS window
  * @frame_counter: input buffer counter for current one second FPS window
  * @frame_rate: frame rate of current instance
- * @operating_rate: operating rate of current instance
  * @hfi_rc_type: rate control type
  * @enc_raw_width: source image width for encoder instance
  * @enc_raw_height: source image height for encoder instance
@@ -107,7 +106,6 @@ struct iris_inst {
 	u64				last_buf_ns;
 	u32				frame_counter;
 	u32				frame_rate;
-	u32				operating_rate;
 	u32				hfi_rc_type;
 	u32				enc_raw_width;
 	u32				enc_raw_height;
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index cf5ae420f646..d346eb668f6a 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -123,7 +123,6 @@ struct platform_inst_caps {
 	u32 mb_cycles_fw;
 	u32 mb_cycles_fw_vpp;
 	u32 max_frame_rate;
-	u32 max_operating_rate;
 };
 
 enum platform_inst_fw_cap_type {
diff --git a/drivers/media/platform/qcom/iris/iris_platform_milos.h b/drivers/media/platform/qcom/iris/iris_platform_milos.h
index 8a45a4aa9e86..e92dac972389 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_milos.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_milos.h
@@ -21,7 +21,6 @@ static struct platform_inst_caps platform_inst_cap_milos = {
 	.mb_cycles_vsp = 25,
 	.mb_cycles_vpp = 200,
 	.max_frame_rate = MILOS_MAXIMUM_FPS,
-	.max_operating_rate = MILOS_MAXIMUM_FPS,
 };
 
 #endif
diff --git a/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h b/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
index 3cfecae80d1e..5cab6df0115c 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
@@ -16,7 +16,6 @@ static struct platform_inst_caps platform_inst_cap_qcs8300 = {
 	.mb_cycles_fw = 326389,
 	.mb_cycles_fw_vpp = 44156,
 	.max_frame_rate = MAXIMUM_FPS,
-	.max_operating_rate = MAXIMUM_FPS,
 };
 
 #endif
diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.h b/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
index 3c9dae995bb2..46fed17b611e 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
@@ -24,7 +24,6 @@ static struct platform_inst_caps platform_inst_cap_sm8550 = {
 	.mb_cycles_fw = 489583,
 	.mb_cycles_fw_vpp = 66234,
 	.max_frame_rate = MAXIMUM_FPS,
-	.max_operating_rate = MAXIMUM_FPS,
 };
 
 #endif
diff --git a/drivers/media/platform/qcom/iris/iris_platform_vpu2.c b/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
index becbe3776544..e31f27fa6a15 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
@@ -58,7 +58,6 @@ static struct platform_inst_caps platform_inst_cap_vpu2 = {
 	.mb_cycles_vsp = 25,
 	.mb_cycles_vpp = 200,
 	.max_frame_rate = MAXIMUM_FPS,
-	.max_operating_rate = MAXIMUM_FPS,
 };
 
 static const struct icc_info iris_icc_info_vpu2[] = {
diff --git a/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c b/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
index 43460b82ae77..aebda855a549 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
@@ -80,7 +80,6 @@ static struct platform_inst_caps platform_inst_cap_ar50lt = {
 	.mb_cycles_fw = 733003,
 	.mb_cycles_fw_vpp = 225975,
 	.max_frame_rate = 120,
-	.max_operating_rate = 120,
 };
 
 const struct iris_platform_data qcm2290_data = {
diff --git a/drivers/media/platform/qcom/iris/iris_utils.c b/drivers/media/platform/qcom/iris/iris_utils.c
index 1096cc4b01c3..6384b4a438b6 100644
--- a/drivers/media/platform/qcom/iris/iris_utils.c
+++ b/drivers/media/platform/qcom/iris/iris_utils.c
@@ -128,13 +128,11 @@ int iris_check_core_mbps(struct iris_inst *inst)
 {
 	struct iris_core *core = inst->core;
 	struct iris_inst *instance;
-	u32 total_mbps = 0, fps = 0;
+	u32 total_mbps = 0;
 
 	mutex_lock(&core->lock);
-	list_for_each_entry(instance, &core->instances, list) {
-		fps = max(instance->frame_rate, instance->operating_rate);
-		total_mbps += iris_get_mbpf(instance) * fps;
-	}
+	list_for_each_entry(instance, &core->instances, list)
+		total_mbps += iris_get_mbpf(instance) * instance->frame_rate;
 	mutex_unlock(&core->lock);
 
 	if (total_mbps > core->iris_platform_data->max_core_mbps)
diff --git a/drivers/media/platform/qcom/iris/iris_venc.c b/drivers/media/platform/qcom/iris/iris_venc.c
index 2f2c56bf9122..0db6ed1056bf 100644
--- a/drivers/media/platform/qcom/iris/iris_venc.c
+++ b/drivers/media/platform/qcom/iris/iris_venc.c
@@ -66,7 +66,6 @@ int iris_venc_inst_init(struct iris_inst *inst)
 	inst->crop.width = DEFAULT_WIDTH;
 	inst->crop.height = DEFAULT_HEIGHT;
 
-	inst->operating_rate = DEFAULT_FPS;
 	inst->frame_rate = DEFAULT_FPS;
 
 	inst->enc_raw_width = DEFAULT_WIDTH;
@@ -396,24 +395,17 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
 {
 	struct platform_inst_caps *caps = inst->core->iris_platform_data->inst_caps;
 	struct vb2_queue *src_q = v4l2_m2m_get_src_vq(inst->m2m_ctx);
-	struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(inst->m2m_ctx);
 	struct v4l2_fract *timeperframe = NULL;
 	u32 default_rate = DEFAULT_FPS;
-	bool is_frame_rate = false;
-	u32 fps, max_rate;
+	u32 fps;
 
 	int ret = 0;
 
-	if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
-		timeperframe = &s_parm->parm.output.timeperframe;
-		max_rate = caps->max_operating_rate;
-		s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
-	} else {
-		timeperframe = &s_parm->parm.capture.timeperframe;
-		is_frame_rate = true;
-		max_rate = caps->max_frame_rate;
-		s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
-	}
+	if (s_parm->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		return -EINVAL;
+
+	timeperframe = &s_parm->parm.output.timeperframe;
+	s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
 
 	if (!timeperframe->denominator || !timeperframe->numerator) {
 		if (!timeperframe->numerator)
@@ -426,18 +418,14 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
 	if (!fps)
 		return -EINVAL;
 
-	if (fps > max_rate) {
+	if (fps > caps->max_frame_rate) {
 		ret = -ENOMEM;
 		goto reset_rate;
 	}
 
-	if (is_frame_rate)
-		inst->frame_rate = fps;
-	else
-		inst->operating_rate = fps;
+	inst->frame_rate = fps;
 
-	if ((s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE && vb2_is_streaming(src_q)) ||
-	    (s_parm->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && vb2_is_streaming(dst_q))) {
+	if (vb2_is_streaming(src_q)) {
 		ret = iris_check_core_mbpf(inst);
 		if (ret)
 			goto reset_rate;
@@ -449,12 +437,8 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
 	return 0;
 
 reset_rate:
-	if (ret) {
-		if (is_frame_rate)
-			inst->frame_rate = default_rate;
-		else
-			inst->operating_rate = default_rate;
-	}
+	if (ret)
+		inst->frame_rate = default_rate;
 
 	return ret;
 }
@@ -463,17 +447,10 @@ int iris_venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
 {
 	struct v4l2_fract *timeperframe = NULL;
 
-	if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
-		timeperframe = &s_parm->parm.output.timeperframe;
-		timeperframe->numerator = 1;
-		timeperframe->denominator = inst->operating_rate;
-		s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
-	} else {
-		timeperframe = &s_parm->parm.capture.timeperframe;
-		timeperframe->numerator = 1;
-		timeperframe->denominator = inst->frame_rate;
-		s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
-	}
+	timeperframe = &s_parm->parm.output.timeperframe;
+	timeperframe->numerator = 1;
+	timeperframe->denominator = inst->frame_rate;
+	s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
 
 	return 0;
 }

-- 
2.34.1


      parent reply	other threads:[~2026-11-17 15:21 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 15:21 [PATCH 0/2] media: qcom: iris: gen2 encoder picture type subscription and operating rate fixups Neil Armstrong
2026-11-17 15:21 ` [PATCH 1/2] media: qcom: iris: gen2: subscribe to the picture type on the encoder instance Neil Armstrong
2026-11-17 15:21 ` Neil Armstrong [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261117-topic-iris-encoder-fixes-v1-2-169ce990d12d@linaro.org \
    --to=neil.armstrong@linaro.org \
    --cc=abhinav.kumar@linux.dev \
    --cc=bod@kernel.org \
    --cc=dikshita.agarwal@oss.qualcomm.com \
    --cc=linux-arm-msm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=mchehab@kernel.org \
    --cc=vikash.garodia@oss.qualcomm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®