From: Daniel Golle <daniel@makrotopia.org>
To: Detlev Casanova <detlev.casanova@collabora.com>,
Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>,
Mauro Carvalho Chehab <mchehab@kernel.org>,
Heiko Stuebner <heiko@sntech.de>,
Nicolas Dufresne <nicolas.dufresne@collabora.com>,
Alex Bee <knaerzche@gmail.com>,
Hans Verkuil <hverkuil+cisco@kernel.org>,
Jonas Karlman <jonas@kwiboo.se>,
Sebastian Fricke <sebastian.fricke@collabora.com>,
linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
Subject: [PATCH 3/7] media: rkvdec: hevc: bound the RPS reads by the control size
Date: Sat, 14 Nov 2026 10:22:57 +0000 [thread overview]
Message-ID: <bb9c44d0f56221874b105394169e5c377ef21b42.1794645268.git.daniel@makrotopia.org> (raw)
In-Reply-To: <cover.1794645268.git.daniel@makrotopia.org>
The RPS helpers read num_short_term_ref_pic_sets elements of the
EXT_SPS_ST_RPS control and num_long_term_ref_pics_sps elements of the
EXT_SPS_LT_RPS control, however many elements userspace passed. Beyond
the passed elements they read stale entries of the control's array,
and past its allocation once the SPS counts more sets than the control
has ever held, as the control framework sizes a dynamic array for the
largest element count it has seen. The short-term sets read from there
are unchecked, and a num_negative_pics or num_positive_pics above 16
overflows the calculated sets, the stack arrays of
st_ref_pic_set_prediction() and the hardware RPS table.
A userspace V4L2 client that can open the RKVDEC m2m decoder can set an
SPS whose counts exceed the number of elements it passed in these
controls, so the reads run past the allocation.
Use the controls only when they hold at least as many elements as the
SPS counts, and treat them as not set otherwise in both back ends.
Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant")
Cc: stable@vger.kernel.org
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
---
.../platform/rockchip/rkvdec/rkvdec-hevc-common.c | 12 ++++++------
.../platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c | 4 ++--
.../platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c | 4 ++--
3 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
index c249e3f16f42c..797a16517ed59 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
@@ -448,18 +448,18 @@ void rkvdec_hevc_run_preamble(struct rkvdec_ctx *ctx,
V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
run->scaling_matrix = ctrl ? ctrl->p_cur.p : NULL;
+ run->ext_sps_st_rps = NULL;
if (ctx->has_sps_st_rps) {
ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS);
- run->ext_sps_st_rps = ctrl ? ctrl->p_cur.p : NULL;
- } else {
- run->ext_sps_st_rps = NULL;
+ if (ctrl && ctrl->elems >= run->sps->num_short_term_ref_pic_sets)
+ run->ext_sps_st_rps = ctrl->p_cur.p;
}
+ run->ext_sps_lt_rps = NULL;
if (ctx->has_sps_lt_rps) {
ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS);
- run->ext_sps_lt_rps = ctrl ? ctrl->p_cur.p : NULL;
- } else {
- run->ext_sps_lt_rps = NULL;
+ if (ctrl && ctrl->elems >= run->sps->num_long_term_ref_pics_sps)
+ run->ext_sps_lt_rps = ctrl->p_cur.p;
}
}
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c
index e1936e87f45be..fea7ff48c0dce 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c
@@ -604,8 +604,8 @@ static int rkvdec_hevc_run(struct rkvdec_ctx *ctx)
* On vdpu381, not setting the long and short term ref sets will just output wrong frames.
* Let's just warn about it and let the decoder run anyway.
*/
- if ((!ctx->has_sps_lt_rps && run.sps->num_long_term_ref_pics_sps) ||
- (!ctx->has_sps_st_rps && run.sps->num_short_term_ref_pic_sets)) {
+ if ((!run.ext_sps_lt_rps && run.sps->num_long_term_ref_pics_sps) ||
+ (!run.ext_sps_st_rps && run.sps->num_short_term_ref_pic_sets)) {
dev_warn_ratelimited(rkvdec->dev, "Long and short term RPS not set\n");
} else {
rkvdec_hevc_assemble_hw_rps(&run, &tbl->rps, &hevc_ctx->st_cache);
diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c
index e9d8b590f0d58..c409d21c08598 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c
@@ -550,8 +550,8 @@ static int rkvdec_hevc_run(struct rkvdec_ctx *ctx)
* To be on the safe side for this new v4l2 control, write an error in the log and mark
* the buffer as failed by returning an error here.
*/
- if ((!ctx->has_sps_lt_rps && run.sps->num_long_term_ref_pics_sps) ||
- (!ctx->has_sps_st_rps && run.sps->num_short_term_ref_pic_sets)) {
+ if ((!run.ext_sps_lt_rps && run.sps->num_long_term_ref_pics_sps) ||
+ (!run.ext_sps_st_rps && run.sps->num_short_term_ref_pic_sets)) {
dev_err_ratelimited(rkvdec->dev, "Long and short term RPS not set\n");
rkvdec_run_postamble(ctx, &run.base);
return -EINVAL;
--
2.56.0
next prev parent reply other threads:[~2026-11-14 10:23 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-14 10:22 [PATCH 0/7] media: rkvdec: Fix the HEVC reference picture set handling Daniel Golle
2026-11-14 10:22 ` [PATCH 1/7] media: rkvdec: hevc: apply request controls first Daniel Golle
2026-11-14 10:22 ` [PATCH 2/7] media: rkvdec: vdpu383: complete refused HEVC requests Daniel Golle
2026-11-14 10:22 ` Daniel Golle [this message]
2026-11-16 13:49 ` [PATCH 3/7] media: rkvdec: hevc: bound the RPS reads by the control size Nicolas Dufresne
2026-11-20 23:17 ` Daniel Golle
2026-11-20 23:23 ` Daniel Golle
2026-11-14 10:23 ` [PATCH 5/7] media: rkvdec: vdpu381: refuse HEVC jobs without RPS sets Daniel Golle
2026-11-14 10:23 ` [PATCH 6/7] media: rkvdec: hevc: cache every short-term RPS set Daniel Golle
2026-11-14 10:23 ` [PATCH 7/7] media: rkvdec: hevc: check the RPS scratch allocation Daniel Golle
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bb9c44d0f56221874b105394169e5c377ef21b42.1794645268.git.daniel@makrotopia.org \
--to=daniel@makrotopia.org \
--cc=detlev.casanova@collabora.com \
--cc=ezequiel@vanguardiasur.com.ar \
--cc=heiko@sntech.de \
--cc=hverkuil+cisco@kernel.org \
--cc=jonas@kwiboo.se \
--cc=knaerzche@gmail.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rockchip@lists.infradead.org \
--cc=mchehab@kernel.org \
--cc=nicolas.dufresne@collabora.com \
--cc=sebastian.fricke@collabora.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®