mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Neil Armstrong <neil.armstrong@linaro.org>
To: Vikash Garodia <vikash.garodia@oss.qualcomm.com>,
	Dikshita Agarwal <dikshita.agarwal@oss.qualcomm.com>,
	Abhinav Kumar <abhinav.kumar@linux.dev>,
	Bryan O'Donoghue <bod@kernel.org>,
	Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/2] media: qcom: iris: venc: drop operating rate to match spec
Date: Wed, 18 Nov 2026 09:51:02 +0100	[thread overview]
Message-ID: <3ea996f9-cfd6-42ba-83ad-39eca5b06a2c@linaro.org> (raw)
In-Reply-To: <20261117-topic-iris-encoder-fixes-v1-2-169ce990d12d@linaro.org>

On 11/17/26 16:21, Neil Armstrong wrote:
> Per the V4L2 Stateful Video Encoder UAPI documentation:
>    Set the raw frame interval on the OUTPUT queue via VIDIOC_S_PARM().
>    This also sets the coded frame interval on the CAPTURE queue to the same value.
> 
> This implies setting the frame interval on the OUTPUT queue _should_ be
> supported as a single operation, and setting the frame interval on the
> CAPTURE queue can be considered as optional since the S_PARM on
> the OUTPUT queue will set the same frame interval on the CAPTURE
> queue.
> 
> The current iris implementation maps:
>   - inst->operating_rate to OUTPUT
>   - inst->frame_rate to CAPTURE
> 
> Only inst->frame_rate is set to the firmware, and inst->operating_rate
> is used in combination with inst->frame_rate to calculate
> the entire iris bandwidth in iris_check_core_mbps() plus
> returing the frame interval with G_PARM on the CAPTURE queue.
> 
> So per the Documentation, the two major V4L2 Stateful Video Encoder
> client implementation (FFMpeg & GStreamer) only calls S_PARM
> on the OUTPUT queue, leaving inst->frame_rate to DEFAULT_FPS,
> and only setting inst->operating_rate.
> 
> Since inst->operating_rate is currently useless and breaks the
> conformance, let's just remove operating_rate entirely
> and just set inst->frame_rate only with S_PARM on the OUTPUT
> queue.
> 
> The two rate variables were probably added to allow scaling
> the iris OPP to allow faster encode speed, but wasn't yet
> properly implemented and breaks the S_PARM compliance.

Sashiko reported I should add:

Fixes: 4ff586ff28e3 ("media: iris: Add support for G/S_PARM for encoder video device")

> 
> Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org>
> ---
>   drivers/media/platform/qcom/iris/iris_instance.h   |  2 -
>   .../platform/qcom/iris/iris_platform_common.h      |  1 -
>   .../media/platform/qcom/iris/iris_platform_milos.h |  1 -
>   .../platform/qcom/iris/iris_platform_qcs8300.h     |  1 -
>   .../platform/qcom/iris/iris_platform_sm8550.h      |  1 -
>   .../media/platform/qcom/iris/iris_platform_vpu2.c  |  1 -
>   .../platform/qcom/iris/iris_platform_vpu_ar50lt.c  |  1 -
>   drivers/media/platform/qcom/iris/iris_utils.c      |  8 ++--
>   drivers/media/platform/qcom/iris/iris_venc.c       | 53 ++++++----------------
>   9 files changed, 18 insertions(+), 51 deletions(-)
> 
> diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h
> index 771269e705d9..8eaeae5aff5f 100644
> --- a/drivers/media/platform/qcom/iris/iris_instance.h
> +++ b/drivers/media/platform/qcom/iris/iris_instance.h
> @@ -59,7 +59,6 @@ struct iris_hfi_session_ops;
>    * @last_buf_ns: start time of received input buffer for current one second FPS window
>    * @frame_counter: input buffer counter for current one second FPS window
>    * @frame_rate: frame rate of current instance
> - * @operating_rate: operating rate of current instance
>    * @hfi_rc_type: rate control type
>    * @enc_raw_width: source image width for encoder instance
>    * @enc_raw_height: source image height for encoder instance
> @@ -107,7 +106,6 @@ struct iris_inst {
>   	u64				last_buf_ns;
>   	u32				frame_counter;
>   	u32				frame_rate;
> -	u32				operating_rate;
>   	u32				hfi_rc_type;
>   	u32				enc_raw_width;
>   	u32				enc_raw_height;
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
> index cf5ae420f646..d346eb668f6a 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_common.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
> @@ -123,7 +123,6 @@ struct platform_inst_caps {
>   	u32 mb_cycles_fw;
>   	u32 mb_cycles_fw_vpp;
>   	u32 max_frame_rate;
> -	u32 max_operating_rate;
>   };
>   
>   enum platform_inst_fw_cap_type {
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_milos.h b/drivers/media/platform/qcom/iris/iris_platform_milos.h
> index 8a45a4aa9e86..e92dac972389 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_milos.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_milos.h
> @@ -21,7 +21,6 @@ static struct platform_inst_caps platform_inst_cap_milos = {
>   	.mb_cycles_vsp = 25,
>   	.mb_cycles_vpp = 200,
>   	.max_frame_rate = MILOS_MAXIMUM_FPS,
> -	.max_operating_rate = MILOS_MAXIMUM_FPS,
>   };
>   
>   #endif
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h b/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
> index 3cfecae80d1e..5cab6df0115c 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_qcs8300.h
> @@ -16,7 +16,6 @@ static struct platform_inst_caps platform_inst_cap_qcs8300 = {
>   	.mb_cycles_fw = 326389,
>   	.mb_cycles_fw_vpp = 44156,
>   	.max_frame_rate = MAXIMUM_FPS,
> -	.max_operating_rate = MAXIMUM_FPS,
>   };
>   
>   #endif
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.h b/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
> index 3c9dae995bb2..46fed17b611e 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
> +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.h
> @@ -24,7 +24,6 @@ static struct platform_inst_caps platform_inst_cap_sm8550 = {
>   	.mb_cycles_fw = 489583,
>   	.mb_cycles_fw_vpp = 66234,
>   	.max_frame_rate = MAXIMUM_FPS,
> -	.max_operating_rate = MAXIMUM_FPS,
>   };
>   
>   #endif
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_vpu2.c b/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
> index becbe3776544..e31f27fa6a15 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_vpu2.c
> @@ -58,7 +58,6 @@ static struct platform_inst_caps platform_inst_cap_vpu2 = {
>   	.mb_cycles_vsp = 25,
>   	.mb_cycles_vpp = 200,
>   	.max_frame_rate = MAXIMUM_FPS,
> -	.max_operating_rate = MAXIMUM_FPS,
>   };
>   
>   static const struct icc_info iris_icc_info_vpu2[] = {
> diff --git a/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c b/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
> index 43460b82ae77..aebda855a549 100644
> --- a/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
> +++ b/drivers/media/platform/qcom/iris/iris_platform_vpu_ar50lt.c
> @@ -80,7 +80,6 @@ static struct platform_inst_caps platform_inst_cap_ar50lt = {
>   	.mb_cycles_fw = 733003,
>   	.mb_cycles_fw_vpp = 225975,
>   	.max_frame_rate = 120,
> -	.max_operating_rate = 120,
>   };
>   
>   const struct iris_platform_data qcm2290_data = {
> diff --git a/drivers/media/platform/qcom/iris/iris_utils.c b/drivers/media/platform/qcom/iris/iris_utils.c
> index 1096cc4b01c3..6384b4a438b6 100644
> --- a/drivers/media/platform/qcom/iris/iris_utils.c
> +++ b/drivers/media/platform/qcom/iris/iris_utils.c
> @@ -128,13 +128,11 @@ int iris_check_core_mbps(struct iris_inst *inst)
>   {
>   	struct iris_core *core = inst->core;
>   	struct iris_inst *instance;
> -	u32 total_mbps = 0, fps = 0;
> +	u32 total_mbps = 0;
>   
>   	mutex_lock(&core->lock);
> -	list_for_each_entry(instance, &core->instances, list) {
> -		fps = max(instance->frame_rate, instance->operating_rate);
> -		total_mbps += iris_get_mbpf(instance) * fps;
> -	}
> +	list_for_each_entry(instance, &core->instances, list)
> +		total_mbps += iris_get_mbpf(instance) * instance->frame_rate;
>   	mutex_unlock(&core->lock);
>   
>   	if (total_mbps > core->iris_platform_data->max_core_mbps)
> diff --git a/drivers/media/platform/qcom/iris/iris_venc.c b/drivers/media/platform/qcom/iris/iris_venc.c
> index 2f2c56bf9122..0db6ed1056bf 100644
> --- a/drivers/media/platform/qcom/iris/iris_venc.c
> +++ b/drivers/media/platform/qcom/iris/iris_venc.c
> @@ -66,7 +66,6 @@ int iris_venc_inst_init(struct iris_inst *inst)
>   	inst->crop.width = DEFAULT_WIDTH;
>   	inst->crop.height = DEFAULT_HEIGHT;
>   
> -	inst->operating_rate = DEFAULT_FPS;
>   	inst->frame_rate = DEFAULT_FPS;
>   
>   	inst->enc_raw_width = DEFAULT_WIDTH;
> @@ -396,24 +395,17 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
>   {
>   	struct platform_inst_caps *caps = inst->core->iris_platform_data->inst_caps;
>   	struct vb2_queue *src_q = v4l2_m2m_get_src_vq(inst->m2m_ctx);
> -	struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(inst->m2m_ctx);
>   	struct v4l2_fract *timeperframe = NULL;
>   	u32 default_rate = DEFAULT_FPS;
> -	bool is_frame_rate = false;
> -	u32 fps, max_rate;
> +	u32 fps;
>   
>   	int ret = 0;
>   
> -	if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> -		timeperframe = &s_parm->parm.output.timeperframe;
> -		max_rate = caps->max_operating_rate;
> -		s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
> -	} else {
> -		timeperframe = &s_parm->parm.capture.timeperframe;
> -		is_frame_rate = true;
> -		max_rate = caps->max_frame_rate;
> -		s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
> -	}
> +	if (s_parm->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> +		return -EINVAL;

Sashiko reported I should drop the V4L2_FMT_FLAG_ENC_CAP_FRAME_INTERVAL now
S_PARM is not supported on CAPTURE, will drop on v2

> +
> +	timeperframe = &s_parm->parm.output.timeperframe;
> +	s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
>   
>   	if (!timeperframe->denominator || !timeperframe->numerator) {
>   		if (!timeperframe->numerator)
> @@ -426,18 +418,14 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
>   	if (!fps)
>   		return -EINVAL;
>   
> -	if (fps > max_rate) {
> +	if (fps > caps->max_frame_rate) {
>   		ret = -ENOMEM;
>   		goto reset_rate;
>   	}
>   
> -	if (is_frame_rate)
> -		inst->frame_rate = fps;
> -	else
> -		inst->operating_rate = fps;
> +	inst->frame_rate = fps;
>   
> -	if ((s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE && vb2_is_streaming(src_q)) ||
> -	    (s_parm->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && vb2_is_streaming(dst_q))) {
> +	if (vb2_is_streaming(src_q)) {
>   		ret = iris_check_core_mbpf(inst);
>   		if (ret)
>   			goto reset_rate;
> @@ -449,12 +437,8 @@ int iris_venc_s_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
>   	return 0;
>   
>   reset_rate:
> -	if (ret) {
> -		if (is_frame_rate)
> -			inst->frame_rate = default_rate;
> -		else
> -			inst->operating_rate = default_rate;
> -	}
> +	if (ret)
> +		inst->frame_rate = default_rate;
>   
>   	return ret;
>   }
> @@ -463,17 +447,10 @@ int iris_venc_g_param(struct iris_inst *inst, struct v4l2_streamparm *s_parm)
>   {
>   	struct v4l2_fract *timeperframe = NULL;
>   
> -	if (s_parm->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> -		timeperframe = &s_parm->parm.output.timeperframe;
> -		timeperframe->numerator = 1;
> -		timeperframe->denominator = inst->operating_rate;
> -		s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
> -	} else {
> -		timeperframe = &s_parm->parm.capture.timeperframe;
> -		timeperframe->numerator = 1;
> -		timeperframe->denominator = inst->frame_rate;
> -		s_parm->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
> -	}
> +	timeperframe = &s_parm->parm.output.timeperframe;
> +	timeperframe->numerator = 1;
> +	timeperframe->denominator = inst->frame_rate;
> +	s_parm->parm.output.capability = V4L2_CAP_TIMEPERFRAME;

Sashiko reported I should also return -EINVAL on CAPTURE to match S_PARM,
I'll do that on v2.

Neil

>   
>   	return 0;
>   }
> 


      reply	other threads:[~2026-11-18  8:51 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17 15:21 [PATCH 0/2] media: qcom: iris: gen2 encoder picture type subscription and operating rate fixups Neil Armstrong
2026-11-17 15:21 ` [PATCH 1/2] media: qcom: iris: gen2: subscribe to the picture type on the encoder instance Neil Armstrong
2026-11-18  8:49   ` Neil Armstrong
2026-11-17 15:21 ` [PATCH 2/2] media: qcom: iris: venc: drop operating rate to match spec Neil Armstrong
2026-11-18  8:51   ` Neil Armstrong [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3ea996f9-cfd6-42ba-83ad-39eca5b06a2c@linaro.org \
    --to=neil.armstrong@linaro.org \
    --cc=abhinav.kumar@linux.dev \
    --cc=bod@kernel.org \
    --cc=dikshita.agarwal@oss.qualcomm.com \
    --cc=linux-arm-msm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=mchehab@kernel.org \
    --cc=vikash.garodia@oss.qualcomm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®