Re: [RFC PATCH 0/3] Implement Region of Interest(ROI) support.

[Deepa Guthyappa Madivalara <deepa.madivalara@oss.qualcomm.com>]

On 1/15/2026 4:38 PM, Deepa Guthyappa Madivalara wrote:
>
> On 1/15/2026 5:42 AM, Nicolas Dufresne wrote:
>> Hi,
>>
>> Le mercredi 14 janvier 2026 à 14:14 -0800, Deepa Guthyappa Madivalara 
>> a écrit
>>
>>
>> [...]
>>
>>> Thanks for the quick comments.
>>> Qp map for can be too much data to be sent from user space to firmware
>>> via control per frame.
>>> Ex: Avc has mbsize as 16 and the max mbpf iris driver supports is 
>>> 8192x4352.
>>> This would mean 136kb of data (8bit Qp) needs to be transferred for 
>>> each
>>> frame in worst case.
>>> While are still evaluating Qp map option, due to firmware performance
>>> issues we are
>>> gravitating more towards rectangle ROI.
>>> I am not sure if we will need to support the Qp map in the future.
>> Have you read how this is implemented in Vulkan and D3D12 ? Please 
>> have a read:
>>
>> - Vulkan Video, see quantizationMapTexelSize [0]
>> - D3D, see QPMapRegionPixelsSize [1]
>>
>> [0] 
>> https://docs.vulkan.org/features/latest/features/proposals/VK_KHR_video_encode_quantization_map.html
>> [1] 
>> https://microsoft.github.io/DirectX-Specs/d3d/D3D12_Video_Encoding_Texture_QPMap_DirtyMap_MotionVectors.html
>>
>> Note that D3D also support dirty regions (what you call ROI in this 
>> proposal),
>> with no limits, since these are translated into map by drivers (its a 
>> software
>> feature on top) and motion search hints, that one seems rare.
>>
>> I'm not against having ROI in our API, its common in older chips 
>> designs, but
>> its clearly going away in the long run since most fixed hardware 
>> impose very low
>> region count, which is not usable for modern application. ROI it 
>> trivial to
>> implement on top of QP maps.
>>
>> A typical use case for that is to use lightweight AI or traditional 
>> CV to locate
>> most relevant portion of a video. The result is more like a heat map, 
>> not a set
>> of rectangles. Then we roughly map that in a low granularity QPMap 
>> before
>> encoding. This allow maintaining very low bandwidth, while preserving 
>> the
>> information needed for the heavier processing in the cloud. I'm 
>> including one of
>> the many example of that, this is a talk from Spiideo [2].
>>
>> [2] 
>> https://gstconf.ubicast.tv/videos/region-based-compression-in-gstreamer/
>>
>>
>> regards,
>> Nicolas
> Thank you for the detailed information. I reviewed the documentation
> and agree with your assessment. I am following up with my team to 
> explore this further.
>
> Thanks,
> Deepa
Hi Nicolas,

Thanks for the references. I was able to go through these and speak to
the teams internally. Here is the understandings and proposal for ROI.

To support rectangle QP, userspace will set compound control and
driver will convert this rectangle QP data to firmware understandable QP 
format
(similar to MB based QP format) and send it to firmware.

To support MB based QP, we need more inputs on how we can send 136KB
QP data eg. 8kUHD frame (one byte QP per 16x16MB) from GST / Userspace to
video driver. Can we send using compound control or any other alternate 
approaches
available?

Thanks
Deepa