From: Nicolas Dufresne <nicolas.dufresne@collabora.com>
To: Detlev Casanova <detlev.casanova@collabora.com>,
Mauro Carvalho Chehab <mchehab@kernel.org>,
Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>,
Heiko Stuebner <heiko@sntech.de>,
Hans Verkuil <hverkuil+cisco@kernel.org>,
Jonas Karlman <jonas@kwiboo.se>
Cc: "Sven Püschel" <s.pueschel@pengutronix.de>,
kernel@collabora.com, linux-media@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v2 5/5] media: rkvdec: Add multicore support
Date: Thu, 19 Nov 2026 13:08:54 -0500 [thread overview]
Message-ID: <fcb9cd01bd333b3f92f000d48adf2d00359398d4.camel@collabora.com> (raw)
In-Reply-To: <20260810-rkvdec-multicore-v2-5-986f89d22cdc@collabora.com>
[-- Attachment #1: Type: text/plain, Size: 12720 bytes --]
Hi,
Le lundi 10 août 2026 à 11:22 -0400, Detlev Casanova a écrit :
> Rockchip SoCs like the RK3588 have multiple independent decoder cores
> defined in the device tree as separate nodes sharing the same
> compatible.
> Extend the component-based driver to actually drive all of them from the
> same v4l2 device.
Either combine and add blank between paragraph please.
>
> It uses the v4l2_m2m_set_max_parallel_jobs() function to set the number
> of available cores, making sure that only 1 job per m2m context is
> running at a given time.
> This is crucial for a stateless decoder as each job may depend on the
> completion of previous jobs.
> This avoids adding a complex scheduler that iwould have to ensure that
> all needed reference frames are fully decoded before decoding a frame.
> Instead, cores will work in parallel on different streams.
>
> As each core has its own IOMMU core, buffers must be mapped in each
> core's IOMMU so that any run() call can use any core without having to
> remap everything.
That was done in previous patch.
>
> To do that, we use rockchip iommu domain's iommu devices list.
> With that, one IOMMU domain can be mapped on multiple devices, meaning
> that each call to iommu_map() will flush the new mapping on all devices
> in the list.
> The IOMMU domain that will have all devices in its list is the first
> core's default domain.
>
> Another domain cannot be used because VB2 allocates buffers through the
> DMA engine, which uses iommu_get_dma_domain() to find the domain to map
> buffers through.
>
> The IOMMU restore function can still work as before, but needs to be more
> explicit in what domain to attach the device to.
> That is because detaching the empty domain will reattach the core's default
> domain, which is wrong (except for the first "main" core).
>
> The RCB temporary buffers are allocated in a dedicated SRAM, and each
> core has its own SRAM, so the mapping for each core's SRAM is added in the
> global domain.
>
> Everything else is mapped through the first core's default domain, making
> the driver write the mappings on both IOMMU cores.
>
> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> ---
> .../media/platform/rockchip/rkvdec/rkvdec-rcb.c | 12 +--
> drivers/media/platform/rockchip/rkvdec/rkvdec.c | 104 +++++++++++++++++++--
> drivers/media/platform/rockchip/rkvdec/rkvdec.h | 8 ++
> 3 files changed, 107 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-rcb.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-rcb.c
> index 44df5b6df80c..977e37cf209b 100644
> --- a/drivers/media/platform/rockchip/rkvdec/rkvdec-rcb.c
> +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-rcb.c
> @@ -76,9 +76,8 @@ void rkvdec_free_rcb(struct rkvdec_dev *rkvdec, struct rkvdec_core *core)
> case RKVDEC_ALLOC_SRAM:
> virt_addr = (unsigned long)cfg->rcb_bufs[i].cpu;
>
> - if (iommu_get_domain_for_dev(core->dev))
> - iommu_unmap(iommu_get_domain_for_dev(core->dev),
> - virt_addr, rcb_size);
> + if (rkvdec->iommu_global_domain)
> + iommu_unmap(rkvdec->iommu_global_domain, virt_addr, rcb_size);
So you just wrongly ported it in previous patch and I missed it. Please squash
this in previous commit.
> gen_pool_free(core->sram_pool, virt_addr, rcb_size);
> break;
> case RKVDEC_ALLOC_DMA:
> @@ -134,7 +133,7 @@ int rkvdec_allocate_rcb(struct rkvdec_dev *rkvdec, struct rkvdec_core *core,
>
> /* Try allocating an SRAM buffer */
> if (core->sram_pool) {
> - if (iommu_get_domain_for_dev(core->dev))
> + if (rkvdec->iommu_global_domain)
> rcb_size = ALIGN(rcb_size, SZ_4K);
>
> cpu = gen_pool_dma_zalloc_align(core->sram_pool,
> @@ -144,12 +143,11 @@ int rkvdec_allocate_rcb(struct rkvdec_dev *rkvdec, struct rkvdec_core *core,
> }
>
> /* If an IOMMU is used, map the SRAM address through it */
> - if (cpu && iommu_get_domain_for_dev(core->dev)) {
> + if (cpu && rkvdec->iommu_global_domain) {
> unsigned long virt_addr = (unsigned long)cpu;
> phys_addr_t phys_addr = dma;
>
> - ret = iommu_map(iommu_get_domain_for_dev(core->dev),
> - virt_addr, phys_addr,
> + ret = iommu_map(rkvdec->iommu_global_domain, virt_addr, phys_addr,
> rcb_size, IOMMU_READ | IOMMU_WRITE, 0);
> if (ret) {
> gen_pool_free(core->sram_pool,
> diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec.c b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> index d3b13132b751..35850231922e 100644
> --- a/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> @@ -1058,6 +1058,37 @@ static const struct media_device_ops rkvdec_media_ops = {
> .req_queue = v4l2_m2m_request_queue,
> };
>
> +/**
> + * Return a core that is available for decoding or null if no core is found.
> + * The caller should make sure to call release_core() when the core is no longer needed.
> + */
> +static struct rkvdec_core *acquire_core(struct rkvdec_dev *rkvdec, struct rkvdec_ctx *ctx)
> +{
> + struct rkvdec_core *core = NULL;
> +
> + guard(spinlock_irqsave)(&rkvdec->cores_lock);
> +
> + if (rkvdec->available_core_count) {
> + core = rkvdec->available_cores[--rkvdec->available_core_count];
> +
> + // Set the current core's ctx to this ctx
> + core->curr_ctx = ctx;
> + }
> +
> + return core;
> +}
> +
> +/**
> + * Release the core to make it available for a next job.
> + */
> +static void release_core(struct rkvdec_dev *rkvdec, struct rkvdec_core *core)
> +{
> + guard(spinlock_irqsave)(&rkvdec->cores_lock);
> +
> + core->curr_ctx = NULL;
> + rkvdec->available_cores[rkvdec->available_core_count++] = core;
> +}
> +
> static void rkvdec_job_finish_no_pm(struct rkvdec_ctx *ctx,
> enum vb2_buffer_state result)
> {
> @@ -1072,6 +1103,11 @@ static void rkvdec_job_finish_no_pm(struct rkvdec_ctx *ctx,
> ctx->coded_fmt_desc->ops->done(ctx, src_buf, dst_buf, result);
> }
>
> + if (ctx->core) {
> + release_core(ctx->dev, ctx->core);
> + ctx->core = NULL;
> + }
> +
> v4l2_m2m_buf_done_and_job_finish(m2m_dev, m2m_ctx, result);
> }
>
> @@ -1153,8 +1189,9 @@ static void rkvdec_device_run(void *priv)
> if (WARN_ON(!desc))
> return;
>
> - ctx->core = ctx->dev->main_core;
> - ctx->core->curr_ctx = ctx;
> + ctx->core = acquire_core(ctx->dev, ctx);
> + if (WARN_ON(!ctx->core))
> + return;
>
> ret = pm_runtime_resume_and_get(ctx->core->dev);
> if (ret < 0) {
> @@ -1424,6 +1461,7 @@ static void rkvdec_v4l2_cleanup(struct rkvdec_dev *rkvdec)
>
> static void rkvdec_iommu_restore(struct rkvdec_core *core)
> {
> + int ret;
> if (core->empty_domain) {
> /*
> * To rewrite mapping into the attached IOMMU core, attach a new empty domain that
> @@ -1432,8 +1470,14 @@ static void rkvdec_iommu_restore(struct rkvdec_core *core)
> * This is safely done in this interrupt handler to make sure no memory get mapped
> * through the IOMMU while the empty domain is attached.
> */
> - iommu_attach_device(core->empty_domain, core->dev);
> + iommu_detach_device(core->curr_ctx->dev->iommu_global_domain, core->dev);
> + ret = iommu_attach_device(core->empty_domain, core->dev);
> + if (ret)
> + dev_warn(core->dev, "Cannot attach empty domain: %d\n", ret);
> iommu_detach_device(core->empty_domain, core->dev);
> + ret = iommu_attach_device(core->curr_ctx->dev->iommu_global_domain, core->dev);
> + if (ret)
> + dev_warn(core->dev, "Cannot attach global domain: %d\n", ret);
Ah, I made a comment about that in previous patch ... Its a bit confusing ...
> }
> }
>
> @@ -1724,14 +1768,35 @@ static int rkvdec_core_bind(struct device *dev, struct device *master, void *dat
> struct platform_device *pdev = to_platform_device(dev);
> struct rkvdec_core *core = platform_get_drvdata(pdev);
> struct rkvdec_dev *rkvdec = data;
> + int id, ret;
> +
> + id = rkvdec->core_count;
> + core->id = id;
> + rkvdec->cores[id] = core;
> +
> + if (id == 0)
> + rkvdec->main_core = core;
> +
> + if (iommu_get_domain_for_dev(dev)) {
> + if (!rkvdec->iommu_global_domain) {
> + rkvdec->iommu_global_domain = iommu_get_domain_for_dev(dev);
> + if (IS_ERR(rkvdec->iommu_global_domain)) {
You do if() above, and if (IS_ERR()) here. I think the second is correct, the
first is broken.
> + rkvdec->iommu_global_domain = NULL;
> + dev_warn_once(dev, "cannot get global domain\n");
> + }
> + }
>
> - if (rkvdec->main_core)
> - return dev_err_probe(dev, -EBUSY,
> - "rkvdec already has a bound core\n");
> + if (rkvdec->iommu_global_domain) {
> + ret = iommu_attach_device(rkvdec->iommu_global_domain, dev);
> + if (ret)
> + dev_warn(dev, "cannot attach global domain to core %d\n", id);
> + }
> + }
>
> - rkvdec->main_core = core;
> + release_core(rkvdec, core);
> + rkvdec->core_count++;
>
> - dev_info(dev, "Registered core\n");
> + dev_info(dev, "Registered core %d\n", id);
>
> return 0;
> }
> @@ -1900,6 +1965,8 @@ static int rkvdec_bind(struct device *dev)
> if (ret)
> goto err_unbind;
>
> + v4l2_m2m_set_max_parallel_jobs(rkvdec->m2m_dev, rkvdec->core_count);
> +
> return 0;
>
> err_unbind:
> @@ -1910,12 +1977,15 @@ static int rkvdec_bind(struct device *dev)
> static void rkvdec_unbind(struct device *dev)
> {
> struct rkvdec_dev *rkvdec = dev_get_drvdata(dev);
> + int i;
>
> - cancel_delayed_work_sync(&rkvdec->main_core->watchdog_work);
> + for (i = 0; i < rkvdec->core_count; i++)
> + cancel_delayed_work_sync(&rkvdec->cores[i]->watchdog_work);
>
> rkvdec_v4l2_cleanup(rkvdec);
>
> - rkvdec_free_rcb(rkvdec, rkvdec->main_core);
> + for (i = 0; i < rkvdec->core_count; i++)
> + rkvdec_free_rcb(rkvdec, rkvdec->cores[i]);
>
> component_unbind_all(dev, NULL);
> }
> @@ -1932,6 +2002,7 @@ static int rkvdec_probe(struct platform_device *pdev)
> struct component_match *match = NULL;
> struct device_node *core_node;
> struct rkvdec_dev *rkvdec;
> + unsigned int num_cores = 0;
>
> if (!match_desc)
> return dev_err_probe(dev, -ENODEV, "missing platform data\n");
> @@ -1943,6 +2014,7 @@ static int rkvdec_probe(struct platform_device *pdev)
> of_node_get(core_node);
> component_match_add_release(dev, &match, component_release_of,
> component_compare_of, core_node);
> + num_cores++;
> }
>
> if (!match)
> @@ -1953,11 +2025,23 @@ static int rkvdec_probe(struct platform_device *pdev)
> if (!rkvdec)
> return -ENOMEM;
>
> + rkvdec->cores = devm_kcalloc(dev, num_cores, sizeof(*rkvdec->cores),
> + GFP_KERNEL);
> + if (!rkvdec->cores)
> + return -ENOMEM;
> +
> + rkvdec->available_cores = devm_kcalloc(dev, num_cores,
> + sizeof(*rkvdec->available_cores),
> + GFP_KERNEL);
> + if (!rkvdec->available_cores)
> + return -ENOMEM;
> +
> rkvdec->variant = match_desc->data;
> if (!rkvdec->variant)
> return dev_err_probe(dev, -ENODEV, "failed to get match data\n");
>
> mutex_init(&rkvdec->vdev_lock);
> + spin_lock_init(&rkvdec->cores_lock);
>
> dev_set_drvdata(dev, rkvdec);
>
> diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec.h b/drivers/media/platform/rockchip/rkvdec/rkvdec.h
> index 36eadcfcabdb..d5977cef6df7 100644
> --- a/drivers/media/platform/rockchip/rkvdec/rkvdec.h
> +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec.h
> @@ -15,6 +15,7 @@
> #include <linux/videodev2.h>
> #include <linux/wait.h>
> #include <linux/clk.h>
> +#include <linux/spinlock.h>
>
> #include <media/v4l2-ctrls.h>
> #include <media/v4l2-device.h>
> @@ -137,6 +138,7 @@ struct rkvdec_core {
> struct iommu_domain *empty_domain;
> struct rkvdec_rcb_config *rcb_config;
> struct rkvdec_ctx *curr_ctx;
> + int id;
> };
>
> struct rkvdec_dev {
> @@ -146,7 +148,13 @@ struct rkvdec_dev {
> struct v4l2_m2m_dev *m2m_dev;
> struct mutex vdev_lock; /* serializes ioctls */
> const struct rkvdec_variant *variant;
> + struct rkvdec_core **cores;
> + int core_count;
> + struct rkvdec_core **available_cores;
> + unsigned int available_core_count;
> + spinlock_t cores_lock; /* serializes core list access */
> struct rkvdec_core *main_core;
> + struct iommu_domain *iommu_global_domain;
> };
>
> struct rkvdec_ctx {
So if you can avoid the iommu part being into that patch, we could focus on
acquire/release mechanism. I think its overall fine.
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
next prev parent reply other threads:[~2026-11-19 18:08 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-10 15:22 [PATCH v2 0/5] media: rkvdec: Enable multi-core support Detlev Casanova
2026-08-10 15:22 ` [PATCH v2 1/5] media: rkvdec: Keep RCB to the correct size Detlev Casanova
2026-11-19 17:33 ` Nicolas Dufresne
2026-08-10 15:22 ` [PATCH v2 2/5] media: rkvdec: Remove unused need_reset Detlev Casanova
2026-11-19 17:35 ` Nicolas Dufresne
2026-08-10 15:22 ` [PATCH v2 3/5] v4l2: export v4l2_m2m_set_max_parallel_jobs Detlev Casanova
2026-11-19 17:36 ` Nicolas Dufresne
2026-08-10 15:22 ` [PATCH v2 4/5] media: rkvdec: Split into core and master platform drivers Detlev Casanova
2026-11-19 17:58 ` Nicolas Dufresne
2026-08-10 15:22 ` [PATCH v2 5/5] media: rkvdec: Add multicore support Detlev Casanova
2026-11-19 18:08 ` Nicolas Dufresne [this message]
2026-11-13 12:39 ` [PATCH v2 0/5] media: rkvdec: Enable multi-core support Lukas Splavec
2026-11-14 4:50 ` Arun Iyengar
2026-11-14 5:01 ` Arun Iyengar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fcb9cd01bd333b3f92f000d48adf2d00359398d4.camel@collabora.com \
--to=nicolas.dufresne@collabora.com \
--cc=detlev.casanova@collabora.com \
--cc=ezequiel@vanguardiasur.com.ar \
--cc=heiko@sntech.de \
--cc=hverkuil+cisco@kernel.org \
--cc=jonas@kwiboo.se \
--cc=kernel@collabora.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rockchip@lists.infradead.org \
--cc=mchehab@kernel.org \
--cc=s.pueschel@pengutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®