From: Neil Armstrong <neil.armstrong@linaro.org>
To: Vikash Garodia <vikash.garodia@oss.qualcomm.com>,
Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>,
Abhinav Kumar <abhinav.kumar@linux.dev>,
Bryan O'Donoghue <bod@kernel.org>,
Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/2] media: qcom: iris: venc: drop operating rate to match spec
Date: Wed, 18 Nov 2026 09:51:02 +0100 [thread overview]
Message-ID: <3ea996f9-cfd6-42ba-83ad-39eca5b06a2c@linaro.org> (raw)
In-Reply-To: <20261117-topic-iris-encoder-fixes-v1-2-169ce990d12d@linaro.org>
On 11/17/26 16:21, Neil Armstrong wrote:
> Per the V4L2 Stateful Video Encoder UAPI documentation:
> Set the raw frame interval on the OUTPUT queue via VIDIOC_S_PARM().
> This also sets the coded frame interval on the CAPTURE queue to the same value.
>
> This implies setting the frame interval on the OUTPUT queue _should_ be
> supported as a single operation, and setting the frame interval on the
> CAPTURE queue can be considered as optional since the S_PARM on
> the OUTPUT queue will set the same frame interval on the CAPTURE
> queue.
>
> The current iris implementation maps:
> - inst->operating_rate to OUTPUT
> - inst->frame_rate to CAPTURE
>
> Only inst->frame_rate is set to the firmware, and inst->operating_rate
> is used in combination with inst->frame_rate to calculate
> the entire iris bandwidth in iris_check_core_mbps() plus
> returing the frame interval with G_PARM on the CAPTURE queue.
>
> So per the Documentation, the two major V4L2 Stateful Video Encoder
> client implementation (FFMpeg & GStreamer) only calls S_PARM
> on the OUTPUT queue, leaving inst->frame_rate to DEFAULT_FPS,
> and only setting inst->operating_rate.
>
> Since inst->operating_rate is currently useless and breaks the
> conformance, let's just remove operating_rate entirely
> and just set inst->frame_rate only with S_PARM on the OUTPUT
> queue.
>
> The two rate variables were probably added to allow scaling
> the iris OPP to allow faster encode speed, but wasn't yet
> properly implemented and breaks the S_PARM compliance.
Sashiko reported I should add:
Fixes: 4ff586ff28e3 ("media: iris: Add support for G/S_PARM for encoder video device")
>
> Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org>
> ---
> drivers/media/platform/qcom/iris/iris_instance.h | 2 -
> .../platform/qcom/iris/iris_platform_common.h | 1 -
> .../media/platform/qcom/iris/iris_platform_milos.h | 1 -
> .../platform/qcom/iris/iris_platform_qcs8300.h | 1 -
> .../platform/qcom/iris/iris_platform_sm8550.h | 1 -
> .../media/platform/qcom/iris/iris_platform_vpu2.c | 1 -
> .../platform/qcom/iris/iris_platform_vpu_ar50lt.c | 1 -
> drivers/media/platform/qcom/iris/iris_utils.c | 8 ++--
> drivers/media/platform/qcom/iris/iris_venc.c | 53 ++++++----------------
> 9 files changed, 18 insertions(+), 51 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
> index 771269e705d9..8eaeae5aff5f 100644
> --- a/drivers/media/platform/qcom/iris/iris_instance.h
> +++ b/drivers/media/platform/qcom/iris/iris_instance.h
> @@ -59,7 +59,6 @@ struct iris_hfi_session_ops;
> * @last_buf_ns: start time of received input buffer for current one second FPS window
> * @frame_counter: input buffer counter for current one second FPS window
> * @frame_rate: frame rate of current instance
> - * @operating_rate: operating rate of current instance
> * @hfi_rc_type: rate control type
> * @enc_raw_width: source image width for encoder instance
> * @enc_raw_height: source image height for encoder instance
> @@ -107,7 +106,6 @@ struct iris_inst {
> u64 last_buf_ns;
> u32 frame_counter;
> u32 frame_rate;
> - u32 operating_rate;
> u32 hfi_rc_type;
> u32 enc_raw_width;
> u32 enc_raw_height;
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
> index cf5ae420f646..d346eb668f6a 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_common.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
> @@ -123,7 +123,6 @@ struct platform_inst_caps {
> u32 mb_cycles_fw;
> u32 mb_cycles_fw_vpp;
> u32 max_frame_rate;
> - u32 max_operating_rate;
> };
>
> enum platform_inst_fw_cap_type {
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_milos.h b/drivers/media/platform/qcom/iris/iris_platform_milos.h
> index 8a45a4aa9e86..e92dac972389 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_milos.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_milos.h
> @@ -21,7 +21,6 @@ static struct platform_inst_caps platform_inst_cap_milos = {
> .mb_cycles_vsp = 25,
> .mb_cycles_vpp = 200,
> .max_frame_rate = MILOS_MAXIMUM_FPS,
> - .max_operating_rate = MILOS_MAXIMUM_FPS,
> };
>
> #endif
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h b/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
> index 3cfecae80d1e..5cab6df0115c 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
> @@ -16,7 +16,6 @@ static struct platform_inst_caps platform_inst_cap_qcs8300 = {
> .mb_cycles_fw = 326389,
> .mb_cycles_fw_vpp = 44156,
> .max_frame_rate = MAXIMUM_FPS,
> - .max_operating_rate = MAXIMUM_FPS,
> };
>
> #endif
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.h b/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
> index 3c9dae995bb2..46fed17b611e 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
> @@ -24,7 +24,6 @@ static struct platform_inst_caps platform_inst_cap_sm8550 = {
> .mb_cycles_fw = 489583,
> .mb_cycles_fw_vpp = 66234,
> .max_frame_rate = MAXIMUM_FPS,
> - .max_operating_rate = MAXIMUM_FPS,
> };
>
> #endif
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_vpu2.c b/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
> index becbe3776544..e31f27fa6a15 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
> @@ -58,7 +58,6 @@ static struct platform_inst_caps platform_inst_cap_vpu2 = {
> .mb_cycles_vsp = 25,
> .mb_cycles_vpp = 200,
> .max_frame_rate = MAXIMUM_FPS,
> - .max_operating_rate = MAXIMUM_FPS,
> };
>
> static const struct icc_info iris_icc_info_vpu2[] = {
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c b/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
> index 43460b82ae77..aebda855a549 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
> @@ -80,7 +80,6 @@ static struct platform_inst_caps platform_inst_cap_ar50lt = {
> .mb_cycles_fw = 733003,
> .mb_cycles_fw_vpp = 225975,
> .max_frame_rate = 120,
> - .max_operating_rate = 120,
> };
>
> const struct iris_platform_data qcm2290_data = {
> diff --git a/drivers/media/platform/qcom/iris/iris_utils.c b/drivers/media/platform/qcom/iris/iris_utils.c
> index 1096cc4b01c3..6384b4a438b6 100644
> --- a/drivers/media/platform/qcom/iris/iris_utils.c
> +++ b/drivers/media/platform/qcom/iris/iris_utils.c
> @@ -128,13 +128,11 @@ int iris_check_core_mbps(struct iris_inst *inst)
> {
> struct iris_core *core = inst->core;
> struct iris_inst *instance;
> - u32 total_mbps = 0, fps = 0;
> + u32 total_mbps = 0;
>
> mutex_lock(&core->lock);
> - list_for_each_entry(instance, &core->instances, list) {
> - fps = max(instance->frame_rate, instance->operating_rate);
> - total_mbps += iris_get_mbpf(instance) * fps;
> - }
> + list_for_each_entry(instance, &core->instances, list)
> + total_mbps += iris_get_mbpf(instance) * instance->frame_rate;
> mutex_unlock(&core->lock);
>
> if (total_mbps > core->iris_platform_data->max_core_mbps)
> diff --git a/drivers/media/platform/qcom/iris/iris_venc.c b/drivers/media/platform/qcom/iris/iris_venc.c
> index 2f2c56bf9122..0db6ed1056bf 100644
> --- a/drivers/media/platform/qcom/iris/iris_venc.c
> +++ b/drivers/media/platform/qcom/iris/iris_venc.c
> @@ -66,7 +66,6 @@ int iris_venc_inst_init(struct iris_inst *inst)
> inst->crop.width = DEFAULT_WIDTH;
> inst->crop.height = DEFAULT_HEIGHT;
>
> - inst->operating_rate = DEFAULT_FPS;
> inst->frame_rate = DEFAULT_FPS;
>
> inst->enc_raw_width = DEFAULT_WIDTH;
> @@ -396,24 +395,17 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
> {
> struct platform_inst_caps *caps = inst->core->iris_platform_data->inst_caps;
> struct vb2_queue *src_q = v4l2_m2m_get_src_vq(inst->m2m_ctx);
> - struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(inst->m2m_ctx);
> struct v4l2_fract *timeperframe = NULL;
> u32 default_rate = DEFAULT_FPS;
> - bool is_frame_rate = false;
> - u32 fps, max_rate;
> + u32 fps;
>
> int ret = 0;
>
> - if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> - timeperframe = &s_parm->parm.output.timeperframe;
> - max_rate = caps->max_operating_rate;
> - s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
> - } else {
> - timeperframe = &s_parm->parm.capture.timeperframe;
> - is_frame_rate = true;
> - max_rate = caps->max_frame_rate;
> - s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
> - }
> + if (s_parm->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> + return -EINVAL;
Sashiko reported I should drop the V4L2_FMT_FLAG_ENC_CAP_FRAME_INTERVAL now
S_PARM is not supported on CAPTURE, will drop on v2
> +
> + timeperframe = &s_parm->parm.output.timeperframe;
> + s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
>
> if (!timeperframe->denominator || !timeperframe->numerator) {
> if (!timeperframe->numerator)
> @@ -426,18 +418,14 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
> if (!fps)
> return -EINVAL;
>
> - if (fps > max_rate) {
> + if (fps > caps->max_frame_rate) {
> ret = -ENOMEM;
> goto reset_rate;
> }
>
> - if (is_frame_rate)
> - inst->frame_rate = fps;
> - else
> - inst->operating_rate = fps;
> + inst->frame_rate = fps;
>
> - if ((s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE && vb2_is_streaming(src_q)) ||
> - (s_parm->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && vb2_is_streaming(dst_q))) {
> + if (vb2_is_streaming(src_q)) {
> ret = iris_check_core_mbpf(inst);
> if (ret)
> goto reset_rate;
> @@ -449,12 +437,8 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
> return 0;
>
> reset_rate:
> - if (ret) {
> - if (is_frame_rate)
> - inst->frame_rate = default_rate;
> - else
> - inst->operating_rate = default_rate;
> - }
> + if (ret)
> + inst->frame_rate = default_rate;
>
> return ret;
> }
> @@ -463,17 +447,10 @@ int iris_venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
> {
> struct v4l2_fract *timeperframe = NULL;
>
> - if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> - timeperframe = &s_parm->parm.output.timeperframe;
> - timeperframe->numerator = 1;
> - timeperframe->denominator = inst->operating_rate;
> - s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
> - } else {
> - timeperframe = &s_parm->parm.capture.timeperframe;
> - timeperframe->numerator = 1;
> - timeperframe->denominator = inst->frame_rate;
> - s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
> - }
> + timeperframe = &s_parm->parm.output.timeperframe;
> + timeperframe->numerator = 1;
> + timeperframe->denominator = inst->frame_rate;
> + s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
Sashiko reported I should also return -EINVAL on CAPTURE to match S_PARM,
I'll do that on v2.
Neil
>
> return 0;
> }
>
prev parent reply other threads:[~2026-11-18 8:51 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 15:21 [PATCH 0/2] media: qcom: iris: gen2 encoder picture type subscription and operating rate fixups Neil Armstrong
2026-11-17 15:21 ` [PATCH 1/2] media: qcom: iris: gen2: subscribe to the picture type on the encoder instance Neil Armstrong
2026-11-18 8:49 ` Neil Armstrong
2026-11-17 15:21 ` [PATCH 2/2] media: qcom: iris: venc: drop operating rate to match spec Neil Armstrong
2026-11-18 8:51 ` Neil Armstrong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3ea996f9-cfd6-42ba-83ad-39eca5b06a2c@linaro.org \
--to=neil.armstrong@linaro.org \
--cc=abhinav.kumar@linux.dev \
--cc=bod@kernel.org \
--cc=dikshita.agarwal@oss.qualcomm.com \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=vikash.garodia@oss.qualcomm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®