[RFC PATCH 0/3] Implement Region of Interest(ROI) support.

[RFC PATCH 0/3] Implement Region of Interest(ROI) support.

[Deepa Guthyappa Madivalara]

Hi all,

This patch set implements region of interest(ROI) support
for video encoder to be configured as a rectangular
region, and corresponding delta QP parameter. A new compound
control V4L2_CID_MPEG_VIDEO_ENC_ROI which maps to struct
v4l2_ctrl_enc_roi_params is implemented to achieve this.  

I'm sharing this series as an RFC because adding support
in the firmware and framework for testing, gstreamer testing
is still in progress. I would appreciate early feedback on
the design, implementation, and fixes before moving to a
formal submission.

v4l2-ctl -d /dev/video1 --list-ctrls
..
hevc_b_frame_maximum_qp_value 0x00990b8c (int): min=1 max=51 step=1
default=51 value=51 flags=has-min-max
video_encoder_roi_params 0x00990b92 (unknown): type=284
value=unsupported payload type flags=has-payload

Thanks,
Deepa

Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
Deepa Guthyappa Madivalara (3):
      media: uapi: Introduce new control for video encoder ROI
      media: v4l2-core: Add support for video encoder ROI control
      media: iris: Add ROI support framework for video encoder

 .../userspace-api/media/v4l/ext-ctrls-codec.rst    |  7 +++
 drivers/media/platform/qcom/iris/iris_ctrls.c      | 54 +++++++++++++++++++++-
 drivers/media/platform/qcom/iris/iris_ctrls.h      |  1 +
 .../platform/qcom/iris/iris_platform_common.h      |  4 ++
 .../media/platform/qcom/iris/iris_platform_gen2.c  |  8 ++++
 drivers/media/v4l2-core/v4l2-ctrls-core.c          | 14 +++++-
 drivers/media/v4l2-core/v4l2-ctrls-defs.c          |  5 ++
 include/media/v4l2-ctrls.h                         |  1 +
 include/uapi/linux/v4l2-controls.h                 |  1 +
 include/uapi/linux/videodev2.h                     | 17 +++++++
 10 files changed, 110 insertions(+), 2 deletions(-)
---
base-commit: f417b7ffcbef7d76b0d8860518f50dae0e7e5eda
change-id: 20260112-iris_enc_roi-8898f9a2455f

Best regards,
-- 
Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>

[RFC PATCH 1/3] media: uapi: Introduce new control for video encoder ROI

[Deepa Guthyappa Madivalara]

Add compound control, V4L2_CID_MPEG_VIDEO_ENC_ROI, for
video encoder Region of Interest to allow applications to specify
different quality levels for specific regions in video frames. Define
struct v4l2_ctrl_enc_roi_params to hold up to 10 rectangular ROI,
regions and their corresponding delta_qp value (v4l2_roi_param)
that adjust quantization relative to the frame's base value.

This enables use cases like prioritizing quality for faces in video
conferencing or important objects in surveillance footage while reducing
bitrate for less critical areas.

Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
 .../userspace-api/media/v4l/ext-ctrls-codec.rst         |  7 +++++++
 include/media/v4l2-ctrls.h                              |  1 +
 include/uapi/linux/v4l2-controls.h                      |  1 +
 include/uapi/linux/videodev2.h                          | 17 +++++++++++++++++
 4 files changed, 26 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index c8890cb5e00ac05649e6c344c2a6b938b2ec1b24..0eecb46bb356c01411dfc313b92376593bcd86f6 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1668,6 +1668,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
     Codecs need to always use the specified range, rather then a HW custom range.
     Applicable to encoders
 
+``V4L2_CID_MPEG_VIDEO_ENC_ROI (struct)``
+    Defines the control id to configure specific delta QP for one or more
+    rectangular regions of interest. The struct v4l2_ctrl_enc_roi_params
+    is defined to hold up to 10 v4l2_rect regions and their corresponding
+    delta_qp with a range of -31 to 30.
+    Applicable to encoders
+
 .. raw:: latex
 
     \normalsize
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 31fc1bee3797bfe532931889188c8f7a9dedad39..c44fad7f51db45a437dd3287aa16830585ac42f3 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -91,6 +91,7 @@ union v4l2_ctrl_ptr {
 	struct v4l2_ctrl_av1_frame *p_av1_frame;
 	struct v4l2_ctrl_av1_film_grain *p_av1_film_grain;
 	struct v4l2_rect *p_rect;
+	struct v4l2_ctrl_enc_roi_params *p_enc_roi_params;
 	void *p;
 	const void *p_const;
 };
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index f84ed133a6c9b2ddc1aedbd582ddf78cb71f34e5..5f2621365593ee19a7792fb25ea29acf6a7860f1 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -918,6 +918,7 @@ enum v4l2_mpeg_video_av1_level {
 };
 
 #define V4L2_CID_MPEG_VIDEO_AVERAGE_QP  (V4L2_CID_CODEC_BASE + 657)
+#define V4L2_CID_MPEG_VIDEO_ENC_ROI  (V4L2_CID_CODEC_BASE + 658)
 
 /*  MPEG-class control IDs specific to the CX2341x driver as defined by V4L2 */
 #define V4L2_CID_CODEC_CX2341X_BASE				(V4L2_CTRL_CLASS_CODEC | 0x1000)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index add08188f06890182a5c399a223c1ab0a546cae1..18a5ae34842721c2647a7a76365e4d299d2b8a44 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1909,6 +1909,7 @@ struct v4l2_ext_control {
 		struct v4l2_ctrl_av1_film_grain __user *p_av1_film_grain;
 		struct v4l2_ctrl_hdr10_cll_info __user *p_hdr10_cll_info;
 		struct v4l2_ctrl_hdr10_mastering_display __user *p_hdr10_mastering_display;
+		struct v4l2_ctrl_enc_roi_params __user *p_enc_roi_params;
 		void __user *ptr;
 	} __attribute__ ((packed));
 } __attribute__ ((packed));
@@ -1990,6 +1991,8 @@ enum v4l2_ctrl_type {
 	V4L2_CTRL_TYPE_AV1_TILE_GROUP_ENTRY = 0x281,
 	V4L2_CTRL_TYPE_AV1_FRAME	    = 0x282,
 	V4L2_CTRL_TYPE_AV1_FILM_GRAIN	    = 0x283,
+
+	V4L2_CTRL_TYPE_ENC_ROI_PARAMS	    = 0x284,
 };
 
 /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
@@ -2540,6 +2543,20 @@ struct v4l2_streamparm {
 	} parm;
 };
 
+/* Roi format
+ */
+#define VIDEO_MAX_ROI_REGIONS         10
+
+struct v4l2_roi_param {
+	struct v4l2_rect roi_rect;
+	__s32 delta_qp;
+};
+
+struct v4l2_ctrl_enc_roi_params {
+	__u32 num_roi_regions;
+	struct v4l2_roi_param roi_params[VIDEO_MAX_ROI_REGIONS];
+};
+
 /*
  *	E V E N T S
  */

-- 
2.34.1

Re: [RFC PATCH 1/3] media: uapi: Introduce new control for video encoder ROI

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 5379 bytes --]

Hi,

Le mardi 13 janvier 2026 à 12:33 -0800, Deepa Guthyappa Madivalara a écrit :
> Add compound control, V4L2_CID_MPEG_VIDEO_ENC_ROI, for
> video encoder Region of Interest to allow applications to specify
> different quality levels for specific regions in video frames. Define
> struct v4l2_ctrl_enc_roi_params to hold up to 10 rectangular ROI,
> regions and their corresponding delta_qp value (v4l2_roi_param)
> that adjust quantization relative to the frame's base value.
> 
> This enables use cases like prioritizing quality for faces in video
> conferencing or important objects in surveillance footage while reducing
> bitrate for less critical areas.
> 
> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
> ---
>  .../userspace-api/media/v4l/ext-ctrls-codec.rst         |  7 +++++++
>  include/media/v4l2-ctrls.h                              |  1 +
>  include/uapi/linux/v4l2-controls.h                      |  1 +
>  include/uapi/linux/videodev2.h                          | 17 +++++++++++++++++
>  4 files changed, 26 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index c8890cb5e00ac05649e6c344c2a6b938b2ec1b24..0eecb46bb356c01411dfc313b92376593bcd86f6 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -1668,6 +1668,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
>      Codecs need to always use the specified range, rather then a HW custom range.
>      Applicable to encoders
>  
> +``V4L2_CID_MPEG_VIDEO_ENC_ROI (struct)``
> +    Defines the control id to configure specific delta QP for one or more
> +    rectangular regions of interest. The struct v4l2_ctrl_enc_roi_params
> +    is defined to hold up to 10 v4l2_rect regions and their corresponding
> +    delta_qp with a range of -31 to 30.
> +    Applicable to encoders

Any justification for this range ? Also, I believe I've seen hardware support
both delta and absolute values. Since it meant to be generic, some research is
needed. If we delibaritly ignore absolute, perhaps the CID should be named
accordingly ? Something like V4L2_CID_MPEG_VIDEO_ENC__DELTAQP_ROI ?

> +
>  .. raw:: latex
>  
>      \normalsize
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 31fc1bee3797bfe532931889188c8f7a9dedad39..c44fad7f51db45a437dd3287aa16830585ac42f3 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -91,6 +91,7 @@ union v4l2_ctrl_ptr {
>  	struct v4l2_ctrl_av1_frame *p_av1_frame;
>  	struct v4l2_ctrl_av1_film_grain *p_av1_film_grain;
>  	struct v4l2_rect *p_rect;
> +	struct v4l2_ctrl_enc_roi_params *p_enc_roi_params;
>  	void *p;
>  	const void *p_const;
>  };
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index f84ed133a6c9b2ddc1aedbd582ddf78cb71f34e5..5f2621365593ee19a7792fb25ea29acf6a7860f1 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -918,6 +918,7 @@ enum v4l2_mpeg_video_av1_level {
>  };
>  
>  #define V4L2_CID_MPEG_VIDEO_AVERAGE_QP  (V4L2_CID_CODEC_BASE + 657)
> +#define V4L2_CID_MPEG_VIDEO_ENC_ROI  (V4L2_CID_CODEC_BASE + 658)
>  
>  /*  MPEG-class control IDs specific to the CX2341x driver as defined by V4L2 */
>  #define V4L2_CID_CODEC_CX2341X_BASE				(V4L2_CTRL_CLASS_CODEC | 0x1000)
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index add08188f06890182a5c399a223c1ab0a546cae1..18a5ae34842721c2647a7a76365e4d299d2b8a44 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -1909,6 +1909,7 @@ struct v4l2_ext_control {
>  		struct v4l2_ctrl_av1_film_grain __user *p_av1_film_grain;
>  		struct v4l2_ctrl_hdr10_cll_info __user *p_hdr10_cll_info;
>  		struct v4l2_ctrl_hdr10_mastering_display __user *p_hdr10_mastering_display;
> +		struct v4l2_ctrl_enc_roi_params __user *p_enc_roi_params;
>  		void __user *ptr;
>  	} __attribute__ ((packed));
>  } __attribute__ ((packed));
> @@ -1990,6 +1991,8 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_AV1_TILE_GROUP_ENTRY = 0x281,
>  	V4L2_CTRL_TYPE_AV1_FRAME	    = 0x282,
>  	V4L2_CTRL_TYPE_AV1_FILM_GRAIN	    = 0x283,
> +
> +	V4L2_CTRL_TYPE_ENC_ROI_PARAMS	    = 0x284,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> @@ -2540,6 +2543,20 @@ struct v4l2_streamparm {
>  	} parm;
>  };
>  
> +/* Roi format
> + */
> +#define VIDEO_MAX_ROI_REGIONS         10

Let's not hardcode hardware spececific constraints in the API. We have dynamic
arrays now in compount controls, and you can define the maximum dimension and
all.


> +
> +struct v4l2_roi_param {
> +	struct v4l2_rect roi_rect;
> +	__s32 delta_qp;
> +};
> +
> +struct v4l2_ctrl_enc_roi_params {
> +	__u32 num_roi_regions;

With the dynamic arrays, this will not be needed.

cheers,
Nicolas

> +	struct v4l2_roi_param roi_params[VIDEO_MAX_ROI_REGIONS];
> +};
> +
>  /*
>   *	E V E N T S
>   */

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/3] media: uapi: Introduce new control for video encoder ROI

[Deepa Guthyappa Madivalara]

On 1/14/2026 8:14 AM, Nicolas Dufresne wrote:
> Hi,
>
> Le mardi 13 janvier 2026 à 12:33 -0800, Deepa Guthyappa Madivalara a écrit :
>> Add compound control, V4L2_CID_MPEG_VIDEO_ENC_ROI, for
>> video encoder Region of Interest to allow applications to specify
>> different quality levels for specific regions in video frames. Define
>> struct v4l2_ctrl_enc_roi_params to hold up to 10 rectangular ROI,
>> regions and their corresponding delta_qp value (v4l2_roi_param)
>> that adjust quantization relative to the frame's base value.
>>
>> This enables use cases like prioritizing quality for faces in video
>> conferencing or important objects in surveillance footage while reducing
>> bitrate for less critical areas.
>>
>> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
>> ---
>>   .../userspace-api/media/v4l/ext-ctrls-codec.rst         |  7 +++++++
>>   include/media/v4l2-ctrls.h                              |  1 +
>>   include/uapi/linux/v4l2-controls.h                      |  1 +
>>   include/uapi/linux/videodev2.h                          | 17 +++++++++++++++++
>>   4 files changed, 26 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index c8890cb5e00ac05649e6c344c2a6b938b2ec1b24..0eecb46bb356c01411dfc313b92376593bcd86f6 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -1668,6 +1668,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
>>       Codecs need to always use the specified range, rather then a HW custom range.
>>       Applicable to encoders
>>   
>> +``V4L2_CID_MPEG_VIDEO_ENC_ROI (struct)``
>> +    Defines the control id to configure specific delta QP for one or more
>> +    rectangular regions of interest. The struct v4l2_ctrl_enc_roi_params
>> +    is defined to hold up to 10 v4l2_rect regions and their corresponding
>> +    delta_qp with a range of -31 to 30.
>> +    Applicable to encoders
> Any justification for this range ? Also, I believe I've seen hardware support
> both delta and absolute values. Since it meant to be generic, some research is
> needed. If we delibaritly ignore absolute, perhaps the CID should be named
> accordingly ? Something like V4L2_CID_MPEG_VIDEO_ENC__DELTAQP_ROI ?

As per Android ROI API - MediaCodec API QP from the app is an offset QP,
meaning userspace will received offset Qp and it converts it to deltaQp
before passing onto the driver in Android HAL. I have used the same idea.
Delta MbQP = frame QP + Offset Qp. This is clamped to -31 to 30 currently
and set to driver as delta QP, hence I have it as -31 to 30.

Absolute values are mostly for frame QP, I would say. All the 
information out there for ROI
kind of implies to deltaQP, but we could be more precise as well.
Let me know if it is a must to change to CID.

>> +
>>   .. raw:: latex
>>   
>>       \normalsize
>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
>> index 31fc1bee3797bfe532931889188c8f7a9dedad39..c44fad7f51db45a437dd3287aa16830585ac42f3 100644
>> --- a/include/media/v4l2-ctrls.h
>> +++ b/include/media/v4l2-ctrls.h
>> @@ -91,6 +91,7 @@ union v4l2_ctrl_ptr {
>>   	struct v4l2_ctrl_av1_frame *p_av1_frame;
>>   	struct v4l2_ctrl_av1_film_grain *p_av1_film_grain;
>>   	struct v4l2_rect *p_rect;
>> +	struct v4l2_ctrl_enc_roi_params *p_enc_roi_params;
>>   	void *p;
>>   	const void *p_const;
>>   };
>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> index f84ed133a6c9b2ddc1aedbd582ddf78cb71f34e5..5f2621365593ee19a7792fb25ea29acf6a7860f1 100644
>> --- a/include/uapi/linux/v4l2-controls.h
>> +++ b/include/uapi/linux/v4l2-controls.h
>> @@ -918,6 +918,7 @@ enum v4l2_mpeg_video_av1_level {
>>   };
>>   
>>   #define V4L2_CID_MPEG_VIDEO_AVERAGE_QP  (V4L2_CID_CODEC_BASE + 657)
>> +#define V4L2_CID_MPEG_VIDEO_ENC_ROI  (V4L2_CID_CODEC_BASE + 658)
>>   
>>   /*  MPEG-class control IDs specific to the CX2341x driver as defined by V4L2 */
>>   #define V4L2_CID_CODEC_CX2341X_BASE				(V4L2_CTRL_CLASS_CODEC | 0x1000)
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index add08188f06890182a5c399a223c1ab0a546cae1..18a5ae34842721c2647a7a76365e4d299d2b8a44 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -1909,6 +1909,7 @@ struct v4l2_ext_control {
>>   		struct v4l2_ctrl_av1_film_grain __user *p_av1_film_grain;
>>   		struct v4l2_ctrl_hdr10_cll_info __user *p_hdr10_cll_info;
>>   		struct v4l2_ctrl_hdr10_mastering_display __user *p_hdr10_mastering_display;
>> +		struct v4l2_ctrl_enc_roi_params __user *p_enc_roi_params;
>>   		void __user *ptr;
>>   	} __attribute__ ((packed));
>>   } __attribute__ ((packed));
>> @@ -1990,6 +1991,8 @@ enum v4l2_ctrl_type {
>>   	V4L2_CTRL_TYPE_AV1_TILE_GROUP_ENTRY = 0x281,
>>   	V4L2_CTRL_TYPE_AV1_FRAME	    = 0x282,
>>   	V4L2_CTRL_TYPE_AV1_FILM_GRAIN	    = 0x283,
>> +
>> +	V4L2_CTRL_TYPE_ENC_ROI_PARAMS	    = 0x284,
>>   };
>>   
>>   /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
>> @@ -2540,6 +2543,20 @@ struct v4l2_streamparm {
>>   	} parm;
>>   };
>>   
>> +/* Roi format
>> + */
>> +#define VIDEO_MAX_ROI_REGIONS         10
> Let's not hardcode hardware spececific constraints in the API. We have dynamic
> arrays now in compount controls, and you can define the maximum dimension and
> all.
>
>
>> +
>> +struct v4l2_roi_param {
>> +	struct v4l2_rect roi_rect;
>> +	__s32 delta_qp;
>> +};
>> +
>> +struct v4l2_ctrl_enc_roi_params {
>> +	__u32 num_roi_regions;
> With the dynamic arrays, this will not be needed.
>
> cheers,
> Nicolas
>
>> +	struct v4l2_roi_param roi_params[VIDEO_MAX_ROI_REGIONS];
>> +};
>> +
>>   /*
>>    *	E V E N T S
>>    */

Re: [RFC PATCH 1/3] media: uapi: Introduce new control for video encoder ROI

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 2294 bytes --]

Hi,

Le mercredi 14 janvier 2026 à 16:20 -0800, Deepa Guthyappa Madivalara a écrit :
> > > +``V4L2_CID_MPEG_VIDEO_ENC_ROI (struct)``
> > > +    Defines the control id to configure specific delta QP for one or more
> > > +    rectangular regions of interest. The struct v4l2_ctrl_enc_roi_params
> > > +    is defined to hold up to 10 v4l2_rect regions and their corresponding
> > > +    delta_qp with a range of -31 to 30.
> > > +    Applicable to encoders
> > Any justification for this range ? Also, I believe I've seen hardware support
> > both delta and absolute values. Since it meant to be generic, some research is
> > needed. If we delibaritly ignore absolute, perhaps the CID should be named
> > accordingly ? Something like V4L2_CID_MPEG_VIDEO_ENC__DELTAQP_ROI ?
> 
> As per Android ROI API - MediaCodec API QP from the app is an offset QP,
> meaning userspace will received offset Qp and it converts it to deltaQp
> before passing onto the driver in Android HAL. I have used the same idea.
> Delta MbQP = frame QP + Offset Qp. This is clamped to -31 to 30 currently
> and set to driver as delta QP, hence I have it as -31 to 30.
> 
> Absolute values are mostly for frame QP, I would say. All the 
> information out there for ROI
> kind of implies to deltaQP, but we could be more precise as well.
> Let me know if it is a must to change to CID.

That's exactly what I want to avoid, hardcoding Android HAL into V4L2 without
having our own rational and documentation. Also, Android HAL is a much older API
then D3D and Vulkan, and its not as well defined.The second is hardcoding range
for one specific implementation. Since this is codec agnostic, and hardware
agnostic control, I would prefer is defined in a way that it requires no scaling
by the driver. IIRC, some codec have QP values from 0 to 63, so why don't we
allo from -63 to 63 ? The alternative is to let the driver expose its range, but
its a little tricky, you will have to specify when this information is available
in the Stateful Video Encoder spec.

As for the rest, you haven't considered extensibility in your proposal, what if
a non Qualcomm hardware do have features like aboslute QP ? (Hantro/VSI does
btw). How do we add that in a clean way ?

Nicolas

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 1/3] media: uapi: Introduce new control for video encoder ROI

[Dmitry Baryshkov]

On Wed, Jan 14, 2026 at 04:20:52PM -0800, Deepa Guthyappa Madivalara wrote:
> 
> On 1/14/2026 8:14 AM, Nicolas Dufresne wrote:
> > Hi,
> > 
> > Le mardi 13 janvier 2026 à 12:33 -0800, Deepa Guthyappa Madivalara a écrit :
> > > Add compound control, V4L2_CID_MPEG_VIDEO_ENC_ROI, for
> > > video encoder Region of Interest to allow applications to specify
> > > different quality levels for specific regions in video frames. Define
> > > struct v4l2_ctrl_enc_roi_params to hold up to 10 rectangular ROI,
> > > regions and their corresponding delta_qp value (v4l2_roi_param)
> > > that adjust quantization relative to the frame's base value.
> > > 
> > > This enables use cases like prioritizing quality for faces in video
> > > conferencing or important objects in surveillance footage while reducing
> > > bitrate for less critical areas.
> > > 
> > > Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
> > > ---
> > >   .../userspace-api/media/v4l/ext-ctrls-codec.rst         |  7 +++++++
> > >   include/media/v4l2-ctrls.h                              |  1 +
> > >   include/uapi/linux/v4l2-controls.h                      |  1 +
> > >   include/uapi/linux/videodev2.h                          | 17 +++++++++++++++++
> > >   4 files changed, 26 insertions(+)
> > > 
> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > index c8890cb5e00ac05649e6c344c2a6b938b2ec1b24..0eecb46bb356c01411dfc313b92376593bcd86f6 100644
> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > @@ -1668,6 +1668,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
> > >       Codecs need to always use the specified range, rather then a HW custom range.
> > >       Applicable to encoders
> > > +``V4L2_CID_MPEG_VIDEO_ENC_ROI (struct)``
> > > +    Defines the control id to configure specific delta QP for one or more
> > > +    rectangular regions of interest. The struct v4l2_ctrl_enc_roi_params
> > > +    is defined to hold up to 10 v4l2_rect regions and their corresponding
> > > +    delta_qp with a range of -31 to 30.
> > > +    Applicable to encoders
> > Any justification for this range ? Also, I believe I've seen hardware support
> > both delta and absolute values. Since it meant to be generic, some research is
> > needed. If we delibaritly ignore absolute, perhaps the CID should be named
> > accordingly ? Something like V4L2_CID_MPEG_VIDEO_ENC__DELTAQP_ROI ?
> 
> As per Android ROI API - MediaCodec API QP from the app is an offset QP,
> meaning userspace will received offset Qp and it converts it to deltaQp
> before passing onto the driver in Android HAL. I have used the same idea.
> Delta MbQP = frame QP + Offset Qp. This is clamped to -31 to 30 currently
> and set to driver as delta QP, hence I have it as -31 to 30.

Please reference Vulkan Video instead, if possible.

-- 
With best wishes
Dmitry

Re: [RFC PATCH 1/3] media: uapi: Introduce new control for video encoder ROI

[Deepa Guthyappa Madivalara]

On 1/14/2026 8:14 AM, Nicolas Dufresne wrote:
> Hi,
>
> Le mardi 13 janvier 2026 à 12:33 -0800, Deepa Guthyappa Madivalara a écrit :
>> Add compound control, V4L2_CID_MPEG_VIDEO_ENC_ROI, for
>> video encoder Region of Interest to allow applications to specify
>> different quality levels for specific regions in video frames. Define
>> struct v4l2_ctrl_enc_roi_params to hold up to 10 rectangular ROI,
>> regions and their corresponding delta_qp value (v4l2_roi_param)
>> that adjust quantization relative to the frame's base value.
>>
>> This enables use cases like prioritizing quality for faces in video
>> conferencing or important objects in surveillance footage while reducing
>> bitrate for less critical areas.
>>
>> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
>> ---
>>   .../userspace-api/media/v4l/ext-ctrls-codec.rst         |  7 +++++++
>>   include/media/v4l2-ctrls.h                              |  1 +
>>   include/uapi/linux/v4l2-controls.h                      |  1 +
>>   include/uapi/linux/videodev2.h                          | 17 +++++++++++++++++
>>   4 files changed, 26 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index c8890cb5e00ac05649e6c344c2a6b938b2ec1b24..0eecb46bb356c01411dfc313b92376593bcd86f6 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -1668,6 +1668,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
>>       Codecs need to always use the specified range, rather then a HW custom range.
>>       Applicable to encoders
>>   
>> +``V4L2_CID_MPEG_VIDEO_ENC_ROI (struct)``
>> +    Defines the control id to configure specific delta QP for one or more
>> +    rectangular regions of interest. The struct v4l2_ctrl_enc_roi_params
>> +    is defined to hold up to 10 v4l2_rect regions and their corresponding
>> +    delta_qp with a range of -31 to 30.
>> +    Applicable to encoders
> Any justification for this range ? Also, I believe I've seen hardware support
> both delta and absolute values. Since it meant to be generic, some research is
> needed. If we delibaritly ignore absolute, perhaps the CID should be named
> accordingly ? Something like V4L2_CID_MPEG_VIDEO_ENC__DELTAQP_ROI ?
>
>> +
>>   .. raw:: latex
>>   
>>       \normalsize
>> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
>> index 31fc1bee3797bfe532931889188c8f7a9dedad39..c44fad7f51db45a437dd3287aa16830585ac42f3 100644
>> --- a/include/media/v4l2-ctrls.h
>> +++ b/include/media/v4l2-ctrls.h
>> @@ -91,6 +91,7 @@ union v4l2_ctrl_ptr {
>>   	struct v4l2_ctrl_av1_frame *p_av1_frame;
>>   	struct v4l2_ctrl_av1_film_grain *p_av1_film_grain;
>>   	struct v4l2_rect *p_rect;
>> +	struct v4l2_ctrl_enc_roi_params *p_enc_roi_params;
>>   	void *p;
>>   	const void *p_const;
>>   };
>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> index f84ed133a6c9b2ddc1aedbd582ddf78cb71f34e5..5f2621365593ee19a7792fb25ea29acf6a7860f1 100644
>> --- a/include/uapi/linux/v4l2-controls.h
>> +++ b/include/uapi/linux/v4l2-controls.h
>> @@ -918,6 +918,7 @@ enum v4l2_mpeg_video_av1_level {
>>   };
>>   
>>   #define V4L2_CID_MPEG_VIDEO_AVERAGE_QP  (V4L2_CID_CODEC_BASE + 657)
>> +#define V4L2_CID_MPEG_VIDEO_ENC_ROI  (V4L2_CID_CODEC_BASE + 658)
>>   
>>   /*  MPEG-class control IDs specific to the CX2341x driver as defined by V4L2 */
>>   #define V4L2_CID_CODEC_CX2341X_BASE				(V4L2_CTRL_CLASS_CODEC | 0x1000)
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index add08188f06890182a5c399a223c1ab0a546cae1..18a5ae34842721c2647a7a76365e4d299d2b8a44 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -1909,6 +1909,7 @@ struct v4l2_ext_control {
>>   		struct v4l2_ctrl_av1_film_grain __user *p_av1_film_grain;
>>   		struct v4l2_ctrl_hdr10_cll_info __user *p_hdr10_cll_info;
>>   		struct v4l2_ctrl_hdr10_mastering_display __user *p_hdr10_mastering_display;
>> +		struct v4l2_ctrl_enc_roi_params __user *p_enc_roi_params;
>>   		void __user *ptr;
>>   	} __attribute__ ((packed));
>>   } __attribute__ ((packed));
>> @@ -1990,6 +1991,8 @@ enum v4l2_ctrl_type {
>>   	V4L2_CTRL_TYPE_AV1_TILE_GROUP_ENTRY = 0x281,
>>   	V4L2_CTRL_TYPE_AV1_FRAME	    = 0x282,
>>   	V4L2_CTRL_TYPE_AV1_FILM_GRAIN	    = 0x283,
>> +
>> +	V4L2_CTRL_TYPE_ENC_ROI_PARAMS	    = 0x284,
>>   };
>>   
>>   /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
>> @@ -2540,6 +2543,20 @@ struct v4l2_streamparm {
>>   	} parm;
>>   };
>>   
>> +/* Roi format
>> + */
>> +#define VIDEO_MAX_ROI_REGIONS         10
> Let's not hardcode hardware spececific constraints in the API. We have dynamic
> arrays now in compount controls, and you can define the maximum dimension and
> all.
>
>
>> +
>> +struct v4l2_roi_param {
>> +	struct v4l2_rect roi_rect;
>> +	__s32 delta_qp;
>> +};
>> +
>> +struct v4l2_ctrl_enc_roi_params {
>> +	__u32 num_roi_regions;
> With the dynamic arrays, this will not be needed.
>
> cheers,
> Nicolas
Sure, I will explore more on this.
>> +	struct v4l2_roi_param roi_params[VIDEO_MAX_ROI_REGIONS];
>> +};
>> +
>>   /*
>>    *	E V E N T S
>>    */

[RFC PATCH 2/3] media: v4l2-core: Add support for video encoder ROI control

[Deepa Guthyappa Madivalara]

Implement V4L2_CTRL_TYPE_ENC_ROI_PARAMS control type with validation
ensuring the number of ROI regions stays within acceptable limits.

Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
 drivers/media/v4l2-core/v4l2-ctrls-core.c | 14 +++++++++++++-
 drivers/media/v4l2-core/v4l2-ctrls-defs.c |  5 +++++
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 209bc05883bb15bc8614970e7b42595aad6dd18f..5695c96b4f2065c5d548a25ebcfd3619e7b6f4d0 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -442,6 +442,9 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
 	case V4L2_CTRL_TYPE_AV1_FILM_GRAIN:
 		pr_cont("AV1_FILM_GRAIN");
 		break;
+	case V4L2_CTRL_TYPE_ENC_ROI_PARAMS:
+		pr_cont("ENCODER_ROI");
+		break;
 	case V4L2_CTRL_TYPE_RECT:
 		pr_cont("(%d,%d)/%ux%u",
 			ptr.p_rect->left, ptr.p_rect->top,
@@ -967,6 +970,7 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
 	struct v4l2_ctrl_hevc_decode_params *p_hevc_decode_params;
 	struct v4l2_area *area;
 	struct v4l2_rect *rect;
+	struct v4l2_ctrl_enc_roi_params *p_enc_roi_params;
 	void *p = ptr.p + idx * ctrl->elem_size;
 	unsigned int i;
 
@@ -1329,7 +1333,12 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
 		if (!rect->width || !rect->height)
 			return -EINVAL;
 		break;
-
+	case V4L2_CTRL_TYPE_ENC_ROI_PARAMS:
+		p_enc_roi_params = p;
+		if (p_enc_roi_params->num_roi_regions > 10 ||
+		    p_enc_roi_params->num_roi_regions < 0)
+			return -EINVAL;
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -2042,6 +2051,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 	case V4L2_CTRL_TYPE_RECT:
 		elem_size = sizeof(struct v4l2_rect);
 		break;
+	case V4L2_CTRL_TYPE_ENC_ROI_PARAMS:
+		elem_size = sizeof(struct v4l2_ctrl_enc_roi_params);
+		break;
 	default:
 		if (type < V4L2_CTRL_COMPOUND_TYPES)
 			elem_size = sizeof(s32);
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index ad41f65374e2396f30f9798235401705af366622..7d36e387d6e9b0e5ee556ea1e48c1b4f0abd084b 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -973,6 +973,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_MPEG_VIDEO_AVERAGE_QP:			return "Average QP Value";
 	case V4L2_CID_FWHT_I_FRAME_QP:				return "FWHT I-Frame QP Value";
 	case V4L2_CID_FWHT_P_FRAME_QP:				return "FWHT P-Frame QP Value";
+	case V4L2_CID_MPEG_VIDEO_ENC_ROI:			return "Video Encoder ROI params ";
 
 	/* VPX controls */
 	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
@@ -1607,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY:
 		*type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY;
 		break;
+	case V4L2_CID_MPEG_VIDEO_ENC_ROI:
+		*type = V4L2_CTRL_TYPE_ENC_ROI_PARAMS;
+		*flags |= V4L2_CTRL_FLAG_HAS_PAYLOAD;
+		break;
 	default:
 		*type = V4L2_CTRL_TYPE_INTEGER;
 		break;

-- 
2.34.1

[RFC PATCH 3/3] media: iris: Add ROI support framework for video encoder

[Deepa Guthyappa Madivalara]

Add ROI support in the iris driver, including control structures
and default parameters. Extend support to set ROI parameters
using compound control V4L2_CTRL_TYPE_ENC_ROI_PARAMS.

Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
---
 drivers/media/platform/qcom/iris/iris_ctrls.c      | 54 +++++++++++++++++++++-
 drivers/media/platform/qcom/iris/iris_ctrls.h      |  1 +
 .../platform/qcom/iris/iris_platform_common.h      |  4 ++
 .../media/platform/qcom/iris/iris_platform_gen2.c  |  8 ++++
 4 files changed, 66 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c
index c0b3a09ad3e3dfb0a47e3603a8089cf61390fda8..2d00df554a64829af550ed43267bf73b6a9c892e 100644
--- a/drivers/media/platform/qcom/iris/iris_ctrls.c
+++ b/drivers/media/platform/qcom/iris/iris_ctrls.c
@@ -98,6 +98,8 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
 		return B_FRAME_QP_H264;
 	case V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP:
 		return B_FRAME_QP_HEVC;
+	case V4L2_CID_MPEG_VIDEO_ENC_ROI:
+		return ROI_PARAMS;
 	default:
 		return INST_FW_CAP_MAX;
 	}
@@ -185,6 +187,8 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
 		return V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP;
 	case B_FRAME_QP_HEVC:
 		return V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP;
+	case ROI_PARAMS:
+		return V4L2_CID_MPEG_VIDEO_ENC_ROI;
 	default:
 		return 0;
 	}
@@ -208,9 +212,14 @@ static int iris_op_s_ctrl(struct v4l2_ctrl *ctrl)
 		return -EINVAL;
 
 	cap[cap_id].flags |= CAP_FLAG_CLIENT_SET;
-
 	inst->fw_caps[cap_id].value = ctrl->val;
 
+	if (inst->fw_caps[cap_id].flags & CAP_FLAG_COMPOUND) {
+		if (cap_id == ROI_PARAMS)
+			inst->fw_caps[cap_id].p_def =
+				(const void *)ctrl->p_new.p_enc_roi_params;
+	}
+
 	if (vb2_is_streaming(q)) {
 		if (cap[cap_id].set)
 			cap[cap_id].set(inst, cap_id);
@@ -223,6 +232,21 @@ static const struct v4l2_ctrl_ops iris_ctrl_ops = {
 	.s_ctrl = iris_op_s_ctrl,
 };
 
+static const struct v4l2_ctrl_enc_roi_params enc_roi_params = {
+	.num_roi_regions = 10,
+	.roi_params = {
+	[0 ... 9] = {
+		.roi_rect = {
+			.left = 0,
+			.top = 0,
+			.width = 0,
+			.height = 0,
+			},
+		.delta_qp = 0,
+		},
+	},
+};
+
 int iris_ctrls_init(struct iris_inst *inst)
 {
 	struct platform_inst_fw_cap *cap = &inst->fw_caps[0];
@@ -263,6 +287,22 @@ int iris_ctrls_init(struct iris_inst *inst)
 						      cap[idx].max,
 						      ~(cap[idx].step_or_mask),
 						      cap[idx].value);
+		} else if (cap[idx].flags & CAP_FLAG_COMPOUND) {
+			if (cap[idx].cap_id == ROI_PARAMS)
+				cap[idx].p_def = &enc_roi_params;
+
+			ctrl = v4l2_ctrl_new_std_compound(&inst->ctrl_handler,
+							  &iris_ctrl_ops,
+							  v4l2_id,
+							  v4l2_ctrl_ptr_create
+							  ((void *)
+							    cap[idx].p_def),
+							  v4l2_ctrl_ptr_create
+							  ((void *)
+							    NULL),
+							  v4l2_ctrl_ptr_create
+							  ((void *)
+							    NULL));
 		} else {
 			ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler,
 						 &iris_ctrl_ops,
@@ -915,3 +955,15 @@ int iris_set_properties(struct iris_inst *inst, u32 plane)
 
 	return 0;
 }
+
+int iris_set_roi_params(struct iris_inst *inst, u32 plane)
+{
+	const struct v4l2_ctrl_enc_roi_params *enc_roi_params;
+	enc_roi_params = inst->fw_caps[ROI_PARAMS].p_def;
+
+	/* Todo: Send HFI prop to firmware
+	 * once support is available
+	 */
+
+	return 0;
+}
diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.h b/drivers/media/platform/qcom/iris/iris_ctrls.h
index 30af333cc4941e737eb1ae83a6944b4192896e23..92ceb9e412635f04787fe92a9e5c6ba03bb7332d 100644
--- a/drivers/media/platform/qcom/iris/iris_ctrls.h
+++ b/drivers/media/platform/qcom/iris/iris_ctrls.h
@@ -33,5 +33,6 @@ int iris_set_max_qp(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_i
 int iris_set_frame_qp(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
 int iris_set_qp_range(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id);
 int iris_set_properties(struct iris_inst *inst, u32 plane);
+int iris_set_roi_params(struct iris_inst *inst, u32 plane);
 
 #endif
diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h
index 8d8cdb56a3c7722c06287d4d10feed14ba2b254c..6fcbdcb11db7785085de8f95b07f93f8a045c9ce 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_common.h
+++ b/drivers/media/platform/qcom/iris/iris_platform_common.h
@@ -6,6 +6,7 @@
 #ifndef __IRIS_PLATFORM_COMMON_H__
 #define __IRIS_PLATFORM_COMMON_H__
 
+#include <media/v4l2-ctrls.h>
 #include <linux/bits.h>
 #include "iris_buffer.h"
 
@@ -143,6 +144,7 @@ enum platform_inst_fw_cap_type {
 	P_FRAME_QP_HEVC,
 	B_FRAME_QP_H264,
 	B_FRAME_QP_HEVC,
+	ROI_PARAMS,
 	INST_FW_CAP_MAX,
 };
 
@@ -154,6 +156,7 @@ enum platform_inst_fw_cap_flags {
 	CAP_FLAG_CLIENT_SET		= BIT(4),
 	CAP_FLAG_BITMASK		= BIT(5),
 	CAP_FLAG_VOLATILE		= BIT(6),
+	CAP_FLAG_COMPOUND		= BIT(7),
 };
 
 struct platform_inst_fw_cap {
@@ -163,6 +166,7 @@ struct platform_inst_fw_cap {
 	s64 step_or_mask;
 	s64 value;
 	u32 hfi_id;
+	const void *p_def;
 	enum platform_inst_fw_cap_flags flags;
 	int (*set)(struct iris_inst *inst,
 		   enum platform_inst_fw_cap_type cap_id);
diff --git a/drivers/media/platform/qcom/iris/iris_platform_gen2.c b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
index c1989240c248601c34b84f508f1b72d72f81260a..eccfe5642d6803fd435787cadc83f878572dbdd5 100644
--- a/drivers/media/platform/qcom/iris/iris_platform_gen2.c
+++ b/drivers/media/platform/qcom/iris/iris_platform_gen2.c
@@ -586,6 +586,14 @@ static const struct platform_inst_fw_cap inst_fw_cap_sm8550_enc[] = {
 		.flags = CAP_FLAG_OUTPUT_PORT,
 		.set = iris_set_u32,
 	},
+	{
+		.cap_id = ROI_PARAMS,
+		.step_or_mask = 1,
+		.p_def = NULL,
+		.flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_COMPOUND |
+			 CAP_FLAG_DYNAMIC_ALLOWED,
+		.set = iris_set_roi_params,
+	},
 };
 
 static struct platform_inst_caps platform_inst_cap_sm8550 = {

-- 
2.34.1

Re: [RFC PATCH 0/3] Implement Region of Interest(ROI) support.

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 3056 bytes --]

Hi,

Le mardi 13 janvier 2026 à 12:33 -0800, Deepa Guthyappa Madivalara a écrit :
> Hi all,
> 
> This patch set implements region of interest(ROI) support
> for video encoder to be configured as a rectangular
> region, and corresponding delta QP parameter. A new compound
> control V4L2_CID_MPEG_VIDEO_ENC_ROI which maps to struct
> v4l2_ctrl_enc_roi_params is implemented to achieve this.  

My very first question will be why ROI rather then QP Map ? Its seems that
modern API such as D3D12 and Vulkan Video aims for QP Map instead of a limited
set of rectangles, while older hardware / firmware have ROI, but since you are
saying that this is not yet implemented in your firmware, I thought it was worth
asking.

The ROI are relatively easy to convert into QP Maps, but the opposite is going
to be a lot less accurate. That being said, the number of ROI can be extremely
limited, at least this is the case for Samsung MFC firmware and Hantro encoders
(no upstream driver yet).

let us know your thought, should we adopt just one, and have driver translate
once HW moved to the new approach ? Should we enventually support both ?

Nicolas

> 
> I'm sharing this series as an RFC because adding support
> in the firmware and framework for testing, gstreamer testing
> is still in progress. I would appreciate early feedback on
> the design, implementation, and fixes before moving to a
> formal submission.
> 
> v4l2-ctl -d /dev/video1 --list-ctrls
> ..
> hevc_b_frame_maximum_qp_value 0x00990b8c (int): min=1 max=51 step=1
> default=51 value=51 flags=has-min-max
> video_encoder_roi_params 0x00990b92 (unknown): type=284
> value=unsupported payload type flags=has-payload
> 
> Thanks,
> Deepa
> 
> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
> ---
> Deepa Guthyappa Madivalara (3):
>       media: uapi: Introduce new control for video encoder ROI
>       media: v4l2-core: Add support for video encoder ROI control
>       media: iris: Add ROI support framework for video encoder
> 
>  .../userspace-api/media/v4l/ext-ctrls-codec.rst    |  7 +++
>  drivers/media/platform/qcom/iris/iris_ctrls.c      | 54
> +++++++++++++++++++++-
>  drivers/media/platform/qcom/iris/iris_ctrls.h      |  1 +
>  .../platform/qcom/iris/iris_platform_common.h      |  4 ++
>  .../media/platform/qcom/iris/iris_platform_gen2.c  |  8 ++++
>  drivers/media/v4l2-core/v4l2-ctrls-core.c          | 14 +++++-
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c          |  5 ++
>  include/media/v4l2-ctrls.h                         |  1 +
>  include/uapi/linux/v4l2-controls.h                 |  1 +
>  include/uapi/linux/videodev2.h                     | 17 +++++++
>  10 files changed, 110 insertions(+), 2 deletions(-)
> ---
> base-commit: f417b7ffcbef7d76b0d8860518f50dae0e7e5eda
> change-id: 20260112-iris_enc_roi-8898f9a2455f
> 
> Best regards,

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 0/3] Implement Region of Interest(ROI) support.

[Deepa Guthyappa Madivalara]

On 1/14/2026 8:08 AM, Nicolas Dufresne wrote:
> Hi,
>
> Le mardi 13 janvier 2026 à 12:33 -0800, Deepa Guthyappa Madivalara a écrit :
>> Hi all,
>>
>> This patch set implements region of interest(ROI) support
>> for video encoder to be configured as a rectangular
>> region, and corresponding delta QP parameter. A new compound
>> control V4L2_CID_MPEG_VIDEO_ENC_ROI which maps to struct
>> v4l2_ctrl_enc_roi_params is implemented to achieve this.
> My very first question will be why ROI rather then QP Map ? Its seems that
> modern API such as D3D12 and Vulkan Video aims for QP Map instead of a limited
> set of rectangles, while older hardware / firmware have ROI, but since you are
> saying that this is not yet implemented in your firmware, I thought it was worth
> asking.
>
> The ROI are relatively easy to convert into QP Maps, but the opposite is going
> to be a lot less accurate. That being said, the number of ROI can be extremely
> limited, at least this is the case for Samsung MFC firmware and Hantro encoders
> (no upstream driver yet).
>
> let us know your thought, should we adopt just one, and have driver translate
> once HW moved to the new approach ? Should we enventually support both ?
>
> Nicolas

Hi Nicolas,

Thanks for the quick comments.
Qp map for can be too much data to be sent from user space to firmware 
via control per frame.
Ex: Avc has mbsize as 16 and the max mbpf iris driver supports is 8192x4352.
This would mean 136kb of data (8bit Qp) needs to be transferred for each 
frame in worst case.
While are still evaluating Qp map option, due to firmware performance 
issues we are
gravitating more towards rectangle ROI.
I am not sure if we will need to support the Qp map in the future.

>> I'm sharing this series as an RFC because adding support
>> in the firmware and framework for testing, gstreamer testing
>> is still in progress. I would appreciate early feedback on
>> the design, implementation, and fixes before moving to a
>> formal submission.
>>
>> v4l2-ctl -d /dev/video1 --list-ctrls
>> ..
>> hevc_b_frame_maximum_qp_value 0x00990b8c (int): min=1 max=51 step=1
>> default=51 value=51 flags=has-min-max
>> video_encoder_roi_params 0x00990b92 (unknown): type=284
>> value=unsupported payload type flags=has-payload
>>
>> Thanks,
>> Deepa
>>
>> Signed-off-by: Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>
>> ---
>> Deepa Guthyappa Madivalara (3):
>>        media: uapi: Introduce new control for video encoder ROI
>>        media: v4l2-core: Add support for video encoder ROI control
>>        media: iris: Add ROI support framework for video encoder
>>
>>   .../userspace-api/media/v4l/ext-ctrls-codec.rst    |  7 +++
>>   drivers/media/platform/qcom/iris/iris_ctrls.c      | 54
>> +++++++++++++++++++++-
>>   drivers/media/platform/qcom/iris/iris_ctrls.h      |  1 +
>>   .../platform/qcom/iris/iris_platform_common.h      |  4 ++
>>   .../media/platform/qcom/iris/iris_platform_gen2.c  |  8 ++++
>>   drivers/media/v4l2-core/v4l2-ctrls-core.c          | 14 +++++-
>>   drivers/media/v4l2-core/v4l2-ctrls-defs.c          |  5 ++
>>   include/media/v4l2-ctrls.h                         |  1 +
>>   include/uapi/linux/v4l2-controls.h                 |  1 +
>>   include/uapi/linux/videodev2.h                     | 17 +++++++
>>   10 files changed, 110 insertions(+), 2 deletions(-)
>> ---
>> base-commit: f417b7ffcbef7d76b0d8860518f50dae0e7e5eda
>> change-id: 20260112-iris_enc_roi-8898f9a2455f
>>
>> Best regards,

Re: [RFC PATCH 0/3] Implement Region of Interest(ROI) support.

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 2077 bytes --]

Hi,

Le mercredi 14 janvier 2026 à 14:14 -0800, Deepa Guthyappa Madivalara a écrit 


[...]

> 
> Thanks for the quick comments.
> Qp map for can be too much data to be sent from user space to firmware 
> via control per frame.
> Ex: Avc has mbsize as 16 and the max mbpf iris driver supports is 8192x4352.
> This would mean 136kb of data (8bit Qp) needs to be transferred for each 
> frame in worst case.
> While are still evaluating Qp map option, due to firmware performance 
> issues we are
> gravitating more towards rectangle ROI.
> I am not sure if we will need to support the Qp map in the future.

Have you read how this is implemented in Vulkan and D3D12 ? Please have a read:

- Vulkan Video, see quantizationMapTexelSize [0]
- D3D, see QPMapRegionPixelsSize [1]

[0] https://docs.vulkan.org/features/latest/features/proposals/VK_KHR_video_encode_quantization_map.html
[1] https://microsoft.github.io/DirectX-Specs/d3d/D3D12_Video_Encoding_Texture_QPMap_DirtyMap_MotionVectors.html

Note that D3D also support dirty regions (what you call ROI in this proposal),
with no limits, since these are translated into map by drivers (its a software
feature on top) and motion search hints, that one seems rare.

I'm not against having ROI in our API, its common in older chips designs, but
its clearly going away in the long run since most fixed hardware impose very low
region count, which is not usable for modern application. ROI it trivial to
implement on top of QP maps.

A typical use case for that is to use lightweight AI or traditional CV to locate
most relevant portion of a video. The result is more like a heat map, not a set
of rectangles. Then we roughly map that in a low granularity QPMap before
encoding. This allow maintaining very low bandwidth, while preserving the
information needed for the heavier processing in the cloud. I'm including one of
the many example of that, this is a talk from Spiideo [2].

[2] https://gstconf.ubicast.tv/videos/region-based-compression-in-gstreamer/


regards,
Nicolas

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH 0/3] Implement Region of Interest(ROI) support.

[Deepa Guthyappa Madivalara]

On 1/15/2026 5:42 AM, Nicolas Dufresne wrote:
> Hi,
>
> Le mercredi 14 janvier 2026 à 14:14 -0800, Deepa Guthyappa Madivalara a écrit
>
>
> [...]
>
>> Thanks for the quick comments.
>> Qp map for can be too much data to be sent from user space to firmware
>> via control per frame.
>> Ex: Avc has mbsize as 16 and the max mbpf iris driver supports is 8192x4352.
>> This would mean 136kb of data (8bit Qp) needs to be transferred for each
>> frame in worst case.
>> While are still evaluating Qp map option, due to firmware performance
>> issues we are
>> gravitating more towards rectangle ROI.
>> I am not sure if we will need to support the Qp map in the future.
> Have you read how this is implemented in Vulkan and D3D12 ? Please have a read:
>
> - Vulkan Video, see quantizationMapTexelSize [0]
> - D3D, see QPMapRegionPixelsSize [1]
>
> [0] https://docs.vulkan.org/features/latest/features/proposals/VK_KHR_video_encode_quantization_map.html
> [1] https://microsoft.github.io/DirectX-Specs/d3d/D3D12_Video_Encoding_Texture_QPMap_DirtyMap_MotionVectors.html
>
> Note that D3D also support dirty regions (what you call ROI in this proposal),
> with no limits, since these are translated into map by drivers (its a software
> feature on top) and motion search hints, that one seems rare.
>
> I'm not against having ROI in our API, its common in older chips designs, but
> its clearly going away in the long run since most fixed hardware impose very low
> region count, which is not usable for modern application. ROI it trivial to
> implement on top of QP maps.
>
> A typical use case for that is to use lightweight AI or traditional CV to locate
> most relevant portion of a video. The result is more like a heat map, not a set
> of rectangles. Then we roughly map that in a low granularity QPMap before
> encoding. This allow maintaining very low bandwidth, while preserving the
> information needed for the heavier processing in the cloud. I'm including one of
> the many example of that, this is a talk from Spiideo [2].
>
> [2] https://gstconf.ubicast.tv/videos/region-based-compression-in-gstreamer/
>
>
> regards,
> Nicolas
Thank you for the detailed information. I reviewed the documentation
and agree with your assessment. I am following up with my team to 
explore this further.

Thanks,
Deepa

Re: [RFC PATCH 0/3] Implement Region of Interest(ROI) support.

[Deepa Guthyappa Madivalara]

On 1/15/2026 4:38 PM, Deepa Guthyappa Madivalara wrote:
>
> On 1/15/2026 5:42 AM, Nicolas Dufresne wrote:
>> Hi,
>>
>> Le mercredi 14 janvier 2026 à 14:14 -0800, Deepa Guthyappa Madivalara 
>> a écrit
>>
>>
>> [...]
>>
>>> Thanks for the quick comments.
>>> Qp map for can be too much data to be sent from user space to firmware
>>> via control per frame.
>>> Ex: Avc has mbsize as 16 and the max mbpf iris driver supports is 
>>> 8192x4352.
>>> This would mean 136kb of data (8bit Qp) needs to be transferred for 
>>> each
>>> frame in worst case.
>>> While are still evaluating Qp map option, due to firmware performance
>>> issues we are
>>> gravitating more towards rectangle ROI.
>>> I am not sure if we will need to support the Qp map in the future.
>> Have you read how this is implemented in Vulkan and D3D12 ? Please 
>> have a read:
>>
>> - Vulkan Video, see quantizationMapTexelSize [0]
>> - D3D, see QPMapRegionPixelsSize [1]
>>
>> [0] 
>> https://docs.vulkan.org/features/latest/features/proposals/VK_KHR_video_encode_quantization_map.html
>> [1] 
>> https://microsoft.github.io/DirectX-Specs/d3d/D3D12_Video_Encoding_Texture_QPMap_DirtyMap_MotionVectors.html
>>
>> Note that D3D also support dirty regions (what you call ROI in this 
>> proposal),
>> with no limits, since these are translated into map by drivers (its a 
>> software
>> feature on top) and motion search hints, that one seems rare.
>>
>> I'm not against having ROI in our API, its common in older chips 
>> designs, but
>> its clearly going away in the long run since most fixed hardware 
>> impose very low
>> region count, which is not usable for modern application. ROI it 
>> trivial to
>> implement on top of QP maps.
>>
>> A typical use case for that is to use lightweight AI or traditional 
>> CV to locate
>> most relevant portion of a video. The result is more like a heat map, 
>> not a set
>> of rectangles. Then we roughly map that in a low granularity QPMap 
>> before
>> encoding. This allow maintaining very low bandwidth, while preserving 
>> the
>> information needed for the heavier processing in the cloud. I'm 
>> including one of
>> the many example of that, this is a talk from Spiideo [2].
>>
>> [2] 
>> https://gstconf.ubicast.tv/videos/region-based-compression-in-gstreamer/
>>
>>
>> regards,
>> Nicolas
> Thank you for the detailed information. I reviewed the documentation
> and agree with your assessment. I am following up with my team to 
> explore this further.
>
> Thanks,
> Deepa
Hi Nicolas,

Thanks for the references. I was able to go through these and speak to
the teams internally. Here is the understandings and proposal for ROI.

To support rectangle QP, userspace will set compound control and
driver will convert this rectangle QP data to firmware understandable QP 
format
(similar to MB based QP format) and send it to firmware.

To support MB based QP, we need more inputs on how we can send 136KB
QP data eg. 8kUHD frame (one byte QP per 16x16MB) from GST / Userspace to
video driver. Can we send using compound control or any other alternate 
approaches
available?

Thanks
Deepa