[PATCH] media: verisilicon: postproc: fix the G1 input height extension

[PATCH] media: verisilicon: postproc: fix the G1 input height extension

[Sergei Khomenkov via B4 Relay]

From: Sergei Khomenkov <shomenkow@gmail.com>

The G1 post-processor takes its input size in macroblocks: the low
bits in swreg72, the high bits in swreg92 (bits 28:26 for bits 11:9
of the width, bits 31:29 for bits 10:8 of the height). This is the
VDPU1 layout in Rockchip's MPP library, whose JPEG decoder writes the
height extension as ((height / 16) & 0x700) >> 8.

hantro_postproc_g1_enable() computes the width extension that way but
writes the height extension as MB_HEIGHT(height >> 8), shifting lines
instead of macroblocks. Every input of 256 to 4080 lines gets an
extension of 1 instead of 0, so the post-processor is told it has 256
more macroblock rows than it has: 286 instead of 30 for 480 lines,
301 instead of 45 for 720 and 324 instead of 68 for 1080. Inputs
below 256 lines are not affected.

Shift the macroblock count, as is done for the width.

The problem was found with an LLM coding assistant while porting
mainline Linux to a Rockchip RK3026 board; the assistant also drafted
this change. Tested on that board (G1, ID 0x6731), decoding H.264
(640x480, 854x480, 1280x720) and VP8 (640x480, 1280x720) to YUYV
through the post-processor, 241 frames, against a software conversion
of the decoder's NV12 output. Without this change only the 4 VP8 key
frames match: a few pixels at the bottom left of the last macroblock
row differ from run to run, and VP8 inter frames propagate them. With
it all 241 frames match. 320x240 is correct either way.

Fixes: 3f6702e1be8a ("media: verisilicon: postproc: 4K support")
Cc: stable@vger.kernel.org
Signed-off-by: Sergei Khomenkov <shomenkow@gmail.com>
---
 drivers/media/platform/verisilicon/hantro_postproc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
index e94d1ba5ef10..ecc6b3137eb0 100644
--- a/drivers/media/platform/verisilicon/hantro_postproc.c
+++ b/drivers/media/platform/verisilicon/hantro_postproc.c
@@ -106,7 +106,7 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
 	HANTRO_PP_REG_WRITE(vpu, orig_width, MB_WIDTH(ctx->dst_fmt.width));
 	HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
 	HANTRO_PP_REG_WRITE(vpu, input_width_ext, MB_WIDTH(ctx->dst_fmt.width) >> 9);
-	HANTRO_PP_REG_WRITE(vpu, input_height_ext, MB_HEIGHT(ctx->dst_fmt.height >> 8));
+	HANTRO_PP_REG_WRITE(vpu, input_height_ext, MB_HEIGHT(ctx->dst_fmt.height) >> 8);
 }
 
 static int down_scale_factor(struct hantro_ctx *ctx)

---
base-commit: 27285b099fbc5cb2ae3aef20f3bbd7b3c88b3005
change-id: 20261104-b4-hantro-pp-1ce1b6891463

Best regards,
-- 
Sergei Khomenkov <shomenkow@gmail.com>

Re: [PATCH] media: verisilicon: postproc: fix the G1 input height extension

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 2771 bytes --]

Le mercredi 04 novembre 2026 à 14:27 +0300, Sergei Khomenkov via B4 Relay a
écrit :
> From: Sergei Khomenkov <shomenkow@gmail.com>
> 
> The G1 post-processor takes its input size in macroblocks: the low
> bits in swreg72, the high bits in swreg92 (bits 28:26 for bits 11:9
> of the width, bits 31:29 for bits 10:8 of the height). This is the
> VDPU1 layout in Rockchip's MPP library, whose JPEG decoder writes the
> height extension as ((height / 16) & 0x700) >> 8.
> 
> hantro_postproc_g1_enable() computes the width extension that way but
> writes the height extension as MB_HEIGHT(height >> 8), shifting lines
> instead of macroblocks. Every input of 256 to 4080 lines gets an
> extension of 1 instead of 0, so the post-processor is told it has 256
> more macroblock rows than it has: 286 instead of 30 for 480 lines,
> 301 instead of 45 for 720 and 324 instead of 68 for 1080. Inputs
> below 256 lines are not affected.
> 
> Shift the macroblock count, as is done for the width.
> 
> The problem was found with an LLM coding assistant while porting
> mainline Linux to a Rockchip RK3026 board; the assistant also drafted
> this change. Tested on that board (G1, ID 0x6731), decoding H.264
> (640x480, 854x480, 1280x720) and VP8 (640x480, 1280x720) to YUYV
> through the post-processor, 241 frames, against a software conversion
> of the decoder's NV12 output. Without this change only the 4 VP8 key
> frames match: a few pixels at the bottom left of the last macroblock
> row differ from run to run, and VP8 inter frames propagate them. With
> it all 241 frames match. 320x240 is correct either way.
> 
> Fixes: 3f6702e1be8a ("media: verisilicon: postproc: 4K support")
> Cc: stable@vger.kernel.org
> Signed-off-by: Sergei Khomenkov <shomenkow@gmail.com>
> ---
>  drivers/media/platform/verisilicon/hantro_postproc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
> index e94d1ba5ef10..ecc6b3137eb0 100644
> --- a/drivers/media/platform/verisilicon/hantro_postproc.c
> +++ b/drivers/media/platform/verisilicon/hantro_postproc.c
> @@ -106,7 +106,7 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
>  	HANTRO_PP_REG_WRITE(vpu, orig_width, MB_WIDTH(ctx->dst_fmt.width));
>  	HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
>  	HANTRO_PP_REG_WRITE(vpu, input_width_ext, MB_WIDTH(ctx->dst_fmt.width) >> 9);
> -	HANTRO_PP_REG_WRITE(vpu, input_height_ext, MB_HEIGHT(ctx->dst_fmt.height >> 8));
> +	HANTRO_PP_REG_WRITE(vpu, input_height_ext, MB_HEIGHT(ctx->dst_fmt.height) >> 8);

Good catch.

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]