Re: [PATCH v3 00/26] Enable H.264/H.265 encoder support and fixes in iris driver common code

[Dikshita Agarwal <quic_dikshita@quicinc.com>]

On 8/21/2025 6:28 PM, Neil Armstrong wrote:
> On 21/08/2025 09:21, Dikshita Agarwal wrote:
>>
>>
>> On 8/20/2025 8:29 PM, Neil Armstrong wrote:
>>> Hi,
>>>
>>> On 20/08/2025 11:07, Dikshita Agarwal wrote:
>>>> Hi All,
>>>>
>>>> This patch series adds support for H.264 and H.265 encoder in iris
>>>> driver and includes a few fixes and cleanup in the common code that were
>>>> identified during encoder bring-up process.
>>>>
>>>> The changes include:
>>>> - Enabling support for H.264 and H.265 encoding.
>>>> - Fixes and improvements in shared componenets used by both encoder and
>>>> decoder paths.
>>>> - Ensuring compatibility and stability with the existing decoder flow.
>>>>
>>>> Changes in v3:
>>>> - Fixed the log when destroying the interanl buffers (Jorge)
>>>> - Updated commit text with issue details in patch 05/25 (Krzysztof)
>>>> - Added a patch to simplify conditional logic in stop handling for hfi
>>>> gen1 (Bryan)
>>>> - Reduced duplicate code while registering video devices (Bryan)
>>>> - Added a fix for try fmt handling in decoder (Self)
>>>> - Fixed the value of max core mbps for qcs8300 (Vikash)
>>>> - Simplied the frame rate handling in driver by using non q16 format and
>>>> converted to q16 when setting to firmware (Vikash)
>>>> - Fixed the issue with bitstream resolution setting to firmware (Neil)
>>>> - Addressed other review comments (Vikash, Bryan)
>>>> - Link to v2:
>>>> https://lore.kernel.org/r/20250813-iris-video-encoder-v2-0-c725ff673078@quicinc.com
>>>>
>>>> Changes in v2:
>>>> - Fixed sparse/coccinnelle issues.
>>>> - Fixed the kernel doc warning.
>>>> - Removed unsupported PEAK_BITRATE property from SM8250.
>>>> - Dropped patch 04/25 to fix quality issue with encoder.
>>>> - Enhanced command handling for encoder to allow start/stop commands.
>>>> - Expanded rate control condition checks to include additional rate
>>>>     control types for HFI Gen2.
>>>> - Updated default value to MAX_QP for all caps related to max QP settings.
>>>> - Add support for INPUT/OUTPUT_BUF_HOST_MAX_COUNT caps for encoder.
>>>> - Link to v1:
>>>> https://lore.kernel.org/r/20250704-iris-video-encoder-v1-0-b6ce24e273cf@quicinc.com
>>>>
>>>> All patches have been tested with v4l2-compliance, v4l2-ctl and
>>>> Gstreamer on SM8250 and SM8550 for encoder, at the same time ensured
>>>> that the existing decoder functionality remains uneffected.
>>>>
>>>> Commands used for V4l2-ctl validation:
>>>>
>>>> v4l2-ctl --verbose
>>>> --set-fmt-video-out=width=1280,height=720,pixelformat=NV12
>>>> --set-selection-output target=crop,top=0,left=0,width=1280,height=720
>>>> --set-fmt-video=pixelformat=H264 --stream-mmap --stream-out-mmap
>>>> --stream-from=/media/cyclists_1280x720_92frames.yuv
>>>> --stream-to=/tmp/cyclists_1280x720_92frames.h264 -d /dev/video1
>>>>
>>>> v4l2-ctl --verbose
>>>> --set-fmt-video-out=width=1280,height=720,pixelformat=NV12
>>>> --set-selection-output target=crop,top=0,left=0,width=1280,height=720
>>>> --set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap
>>>> --stream-from=/media/cyclists_1280x720_92frames.yuv
>>>> --stream-to=/tmp/cyclists_1280x720_92frames.hevc -d /dev/video1
>>>
>>> - I've tested this serie on SM8650, first I failed to get HEVC working with
>>> the following command:
>>>
>>> v4l2-ctl --verbose
>>> --set-fmt-video-out=width=1920,height=1080,pixelformat=NV12
>>> --set-selection-output target=crop,top=0,left=0,width=1920,height=1080
>>> --set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap
>>> --stream-from=Big_Buck_Bunny_1080_10s.yuv
>>> --stream-to=Big_Buck_Bunny_1080_10s.hevc -d /dev/video1
>>>
>>> I got:
>>> qcom-iris aa00000.video-codec: session error received 0x1000005: unknown
>>>
>>> I checked the diff with v2, and reverting this make HEVC encoding work
>>> again:
>>
>> This is strange, we don't see any such issue on SM8550.
>> Could you pls share the full logs.
> 
> I tried again with a 720p stream with your exact command line from the
> cover letter
> and it still fails with HEVC:
> $ v4l2-ctl --verbose
> --set-fmt-video-out=width=1280,height=720,pixelformat=NV12
> --set-selection-output target=crop,top=0,left=0,width=1280,height=720
> --set-fmt-video=pixelformat=HEVC --stream-mmap --stream-out-mmap
> --stream-from=Big_Buck_Bunny_720_10s.yuv
> --stream-to=Big_Buck_Bunny_720_10s.hevc -d /dev/video1
> VIDIOC_QUERYCAP: ok
> VIDIOC_G_FMT: ok
> VIDIOC_S_FMT: ok
> Format Video Capture Multiplanar:
>     Width/Height      : 320/240
>     Pixel Format      : 'HEVC' (HEVC)
>     Field             : None
>     Number of planes  : 1
>     Flags             :
>     Colorspace        : Default
>     Transfer Function : Default
>     YCbCr/HSV Encoding: Default
>     Quantization      : Default
>     Plane 0           :
>        Bytes per Line : 0
>        Size Image     : 245760
> VIDIOC_G_FMT: ok
> VIDIOC_S_FMT: ok
> Format Video Output Multiplanar:
>     Width/Height      : 1280/736
>     Pixel Format      : 'NV12' (Y/UV 4:2:0)
>     Field             : None
>     Number of planes  : 1
>     Flags             :
>     Colorspace        : Default
>     Transfer Function : Default
>     YCbCr/HSV Encoding: Default
>     Quantization      : Default
>     Plane 0           :
>        Bytes per Line : 1280
>        Size Image     : 1413120
> VIDIOC_G_SELECTION: ok
> VIDIOC_S_SELECTION: ok
>         VIDIOC_G_FMT returned 0 (Success)
>         VIDIOC_G_FMT returned 0 (Success)
>         VIDIOC_G_FMT returned 0 (Success)
>         VIDIOC_REQBUFS returned 0 (Success)
>         VIDIOC_QUERYBUF returned 0 (Success)
>         VIDIOC_QUERYBUF returned 0 (Success)
>         VIDIOC_QUERYBUF returned 0 (Success)
>         VIDIOC_QUERYBUF returned 0 (Success)
>         VIDIOC_G_FMT returned 0 (Success)
>         VIDIOC_QUERYBUF returned 0 (Success)
>         VIDIOC_G_FMT returned 0 (Success)
>         VIDIOC_QBUF returned 0 (Success)
>         VIDIOC_QUERYBUF returned 0 (Success)
>         VIDIOC_G_FMT returned 0 (Success)
>         VIDIOC_QBUF returned 0 (Success)
>         VIDIOC_QUERYBUF returned 0 (Success)
>         VIDIOC_G_FMT returned 0 (Success)
>         VIDIOC_QBUF returned 0 (Success)
>         VIDIOC_QUERYBUF returned 0 (Success)
>         VIDIOC_G_FMT returned 0 (Success)
>         VIDIOC_QBUF returned 0 (Success)
>         VIDIOC_STREAMON returned 0 (Success)
> VIDIOC_G_FMT: ok
> VIDIOC_DQBUF: failed: Input/output error
> out dqbuf: 0 seq:      0 bytesused: 0 ts: 0.000000 field: Any
> VIDIOC_DQBUF: failed: Input/output error
> 

Thanks for the logs, but those didn't give any useful info on this issue,
We would firmware logs to check this, as we are not able to reproduce this
issue on SM8550.

Please use below patch to enabled firmware logs and share the debug output.

if you see any issue due to higher logging, you can reduce the paylod value
for HFI_PROP_DEBUG_LOG_LEVEL to 0x00000018.

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.c
b/drivers/media/platform/qcom/iris/iris_hfi_common.c
index 92112eb16c110..b7061faa82ae2 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_common.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_common.c
@@ -87,6 +87,10 @@ int iris_hfi_core_init(struct iris_core *core)
 	if (ret)
 		return ret;

+	ret = hfi_ops->sys_set_debug(core);
+	if (ret)
+		return ret;
+
 	return hfi_ops->sys_interframe_powercollapse(core);
 }

diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h
b/drivers/media/platform/qcom/iris/iris_hfi_common.h
index b51471fb32c70..edda5a531953d 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_common.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h
@@ -108,6 +108,7 @@ struct iris_hfi_prop_type_handle {
 struct iris_hfi_command_ops {
 	int (*sys_init)(struct iris_core *core);
 	int (*sys_image_version)(struct iris_core *core);
+	int (*sys_set_debug)(struct iris_core *core);
 	int (*sys_interframe_powercollapse)(struct iris_core *core);
 	int (*sys_pc_prep)(struct iris_core *core);
 	int (*session_set_config_params)(struct iris_inst *inst, u32 plane);
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
index c97e4910e8528..ec3e59c615a40 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c
@@ -54,6 +54,23 @@ static int iris_hfi_gen2_sys_image_version(struct
iris_core *core)
 	return ret;
 }

+static int iris_hfi_gen2_sys_set_debug(struct iris_core *core)
+{
+	struct iris_hfi_header *hdr;
+	int ret;
+
+	hdr = kzalloc(SYS_IFPC_PKT_SIZE, GFP_KERNEL);
+	if (!hdr)
+		return -ENOMEM;
+
+	iris_hfi_gen2_packet_set_debug(core, hdr);
+	ret = iris_hfi_queue_cmd_write_locked(core, hdr, hdr->size);
+
+	kfree(hdr);
+
+	return ret;
+}
+
 static int iris_hfi_gen2_sys_interframe_powercollapse(struct iris_core *core)
 {
 	struct iris_hfi_header *hdr;
@@ -1170,6 +1187,7 @@ static int
iris_hfi_gen2_session_release_buffer(struct iris_inst *inst, struct i
 static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = {
 	.sys_init = iris_hfi_gen2_sys_init,
 	.sys_image_version = iris_hfi_gen2_sys_image_version,
+	.sys_set_debug = iris_hfi_gen2_sys_set_debug,
 	.sys_interframe_powercollapse = iris_hfi_gen2_sys_interframe_powercollapse,
 	.sys_pc_prep = iris_hfi_gen2_sys_pc_prep,
 	.session_open = iris_hfi_gen2_session_open,
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
index aa1f795f5626c..4f0f4e95652c1 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h
@@ -187,6 +187,9 @@ enum hfi_packet_firmware_flags {
 	HFI_FW_FLAGS_SYSTEM_ERROR		= 0x00000008,
 };

+#define HFI_PROP_DEBUG_CONFIG                                   0x0300000a
+#define HFI_PROP_DEBUG_LOG_LEVEL                                0x0300000b
+
 struct hfi_debug_header {
 	u32 size;
 	u32 debug_level;
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c
b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c
index d77fa29f44fc6..14b0abbb8bc02 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c
@@ -222,6 +222,33 @@ void iris_hfi_gen2_packet_image_version(struct
iris_core *core, struct iris_hfi_
 				    NULL, 0);
 }

+void iris_hfi_gen2_packet_set_debug(struct iris_core *core, struct
iris_hfi_header *hdr)
+{
+	u32 payload = 0;
+
+	iris_hfi_gen2_create_header(hdr, 0, core->header_id++);
+
+	payload = 0;
+	iris_hfi_gen2_create_packet(hdr,
+				    HFI_PROP_DEBUG_CONFIG,
+				    HFI_HOST_FLAGS_NONE,
+				    HFI_PAYLOAD_U32_ENUM,
+				    HFI_PORT_NONE,
+				    core->packet_id++,
+				    &payload,
+				    sizeof(u32));
+
+	payload = 0x0000001c;
+	iris_hfi_gen2_create_packet(hdr,
+				    HFI_PROP_DEBUG_LOG_LEVEL,
+				    HFI_HOST_FLAGS_NONE,
+				    HFI_PAYLOAD_U32_ENUM,
+				    HFI_PORT_NONE,
+				    core->packet_id++,
+				    &payload,
+				    sizeof(u32));
+}
+
 void iris_hfi_gen2_packet_session_command(struct iris_inst *inst, u32
pkt_type,
 					  u32 flags, u32 port, u32 session_id,
 					  u32 payload_type, void *payload,
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h
b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h
index 25b9582349ca1..f44c09af694a0 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h
@@ -121,5 +121,6 @@ void iris_hfi_gen2_packet_session_property(struct
iris_inst *inst,
 void iris_hfi_gen2_packet_sys_interframe_powercollapse(struct iris_core *core,
 						       struct iris_hfi_header *hdr);
 void iris_hfi_gen2_packet_sys_pc_prep(struct iris_core *core, struct
iris_hfi_header *hdr);
+void iris_hfi_gen2_packet_set_debug(struct iris_core *core, struct
iris_hfi_header *hdr);

 #endif
diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
index 2f1f118eae4f6..345e438967ead 100644
--- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
+++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c
@@ -949,7 +949,7 @@ static void iris_hfi_gen2_flush_debug_queue(struct
iris_core *core, u8 *packet)

 		packet[pkt->size] = '\0';
 		log = (u8 *)packet + sizeof(*pkt) + 1;
-		dev_dbg(core->dev, "%s", log);
+		dev_err(core->dev, "%s", log);
 	}
 }

Thanks,
Dikshita