Re: [PATCH 0/4] media: Track v4l2 buffers through an allocator

["Diederik de Haas" <diederik@cknow-tech.com>]

Hi Detlev,

On Wed Sep 16, 2026 at 4:25 PM CEST, Detlev Casanova wrote:
> Hello,
>
> Currently, the only way to track buffers allocated by v4l2 from usespace
> is to use the subsystem available debug information (e.g.:
> /sys/kernel/debug/dma_buf/bufinfo).
> But that information is generic and cannot be matched to a v4l2 driver or
> to a userspace application: It is merely information about the allocation.
>
> Other types of allocations require the developper to find where they are
> exposed and how to link them to their test.
>
> This can become hard to track when mutliple drivers are working at the
> same time.
>
> To improve that, add a small wrapper around buffer allocations to keep
> track of them at the video device level so that we can add debug
> information to them like a name, userspace pid/fd that did the
> allocation,... and expose them to userspace via a debugfs entry.
>
> It currently only supports DMA buffer allocations and adds support for
> VB2 allocations too.
> Other kind of memory tracking can be added later.
> The verisilicon and rkvdec drivers have been ported to use the tracked
> dma alloactions.
>
> Note that this depends on the ftrace support patch series[1] that
> provides fd/pid info in the v4l2_fh struct.
> That series is a bit old, so this is based on an older linux version, but
> that only changes things for the last 2 commits.
>
> A v4l2top utility[2] has been made, to be used in parallel with the
> fdinfo patch series[3], to show the list of active streams with their HW
> and memory usage.
>
> With this, debugfs looks like this when decoding a HEVC 1080p stream with
> rkvdec on rk3588:
>
>   root # cat /sys/kernel/debug/v4l2/fdc38100.video-codec/mem 
>   created-by                      fd              pid             size            label
>   -------------------------------------------------------------------------------------
>   gst-launch-1.0                  7               635             39184           vdpu381-hevc-priv-tbl
>   gst-launch-1.0                  7               635             4177920         cap-00000000ac34392e-7
>   gst-launch-1.0                  7               635             4177920         cap-00000000ac34392e-6
>   gst-launch-1.0                  7               635             4177920         cap-00000000ac34392e-5
>   gst-launch-1.0                  7               635             4177920         cap-00000000ac34392e-4
>   gst-launch-1.0                  7               635             4177920         cap-00000000ac34392e-3
>   gst-launch-1.0                  7               635             4177920         cap-00000000ac34392e-2
>   gst-launch-1.0                  7               635             4177920         cap-00000000ac34392e-1
>   gst-launch-1.0                  7               635             4177920         cap-00000000ac34392e-0
>   gst-launch-1.0                  7               635             3133440         out-000000005c0230f6-1
>   gst-launch-1.0                  7               635             3133440         out-000000005c0230f6-0
>   =====================================================================================
>   Total size: 39729424

I build a kernel with your 3 patch series and I also build the v4l2top utility.
Cool tool :-D 
Especially after I found out I should build the 'upstream' branch ;-)

There were a couple of things I noticed:
1) With 'plain' kernel 7.3-rc3 but also with all my patches included, but
   excluding the patches from this series, I see this:

```
root@nanopc-t6-plus:~# ls -lh /sys/kernel/debug/v4l2/
total 0
drwxr-xr-x 3 root root 0 Sep 20 15:10 fdee0000.hdmi_receiver
```

Which is presumably the reason that with a kernel which also includes this
patch series, I get the following kernel error:
``[   10.849405] debugfs: 'v4l2' already exists in '/'``

I don't think this should generate an error/warning/info message at all,
but just silently dealt with.

Checking v4l2 debug dir again results in this:
```
root@nanopc-t6-plus:~# ls -lh /sys/kernel/debug/v4l2/
total 0
drwxr-xr-x 2 root root 0 Sep 20 15:13 fdb60000.rga
drwxr-xr-x 2 root root 0 Sep 20 15:13 fdb80000.rga
drwxr-xr-x 2 root root 0 Sep 20 15:13 fdc38000.video-codec
drwxr-xr-x 2 root root 0 Sep 20 15:13 fdee0000.hdmi_receiver
```

I noticed my video-codec address was slightly different from yours.

Playing a 1080p BBB x264 video with a patched ffmpeg+mpv, results in this:
```
root@nanopc-t6-plus:~# cat /sys/kernel/debug/v4l2/fdc38000.video-codec/mem
created-by                      fd              pid             size            label
-------------------------------------------------------------------------------------
mpv                             26              2027            4177920         cap-00000000205a3b3a-10
mpv                             26              2027            4177920         cap-00000000205a3b3a-9
mpv                             26              2027            4177920         cap-00000000205a3b3a-8
mpv                             26              2027            16736           vdpu381-h264-priv-tbl
mpv                             26              2027            3133440         out-0000000079002cb1-3
mpv                             26              2027            3133440         out-0000000079002cb1-2
mpv                             26              2027            3133440         out-0000000079002cb1-1
mpv                             26              2027            3133440         out-0000000079002cb1-0
mpv                             26              2027            4177920         cap-00000000205a3b3a-7
mpv                             26              2027            4177920         cap-00000000205a3b3a-6
mpv                             26              2027            4177920         cap-00000000205a3b3a-5
mpv                             26              2027            4177920         cap-00000000205a3b3a-4
mpv                             26              2027            4177920         cap-00000000205a3b3a-3
mpv                             26              2027            4177920         cap-00000000205a3b3a-2
mpv                             26              2027            4177920         cap-00000000205a3b3a-1
mpv                             26              2027            4177920         cap-00000000205a3b3a-0
=====================================================================================
Total size: 58507616
```

And looking at the v4l2top utility, I noticed the following:
1. I see differing indications of the load while it's playing :-D
2. The ``Clock rate`` is always 750.0MHz (or 786431991Hz) (expected I think?)
3. The ``Total Memory`` is always 0B and the ``Memory usage details`` column
   is always empty. That's also the case when I press one of the arrow keys
   which seems to select the entry. Am I missing sth or doing sth wrong?

Cheers,
  Diederik

> [1]: https://lore.kernel.org/all/20260610-v4l2-add-ftrace-v2-0-9756edf72ac1@collabora.com/
> [2]: https://github.com/cazou/v4l2top/tree/upstream
> [3]: https://lore.kernel.org/all/20260706-v4l2-add-fdinfo-v3-0-d556568cf38e@collabora.com/
>
> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
> ---
> Detlev Casanova (4):
>       media: Add a v4l2 memory allocations tracker
>       media: Store v4l2_fh in the vb_queue
>       media: verisilicon: Switch to tracked dma allocations
>       media: rkvdec: Switch to tracked dma allocations
>
>  drivers/media/common/videobuf2/Makefile            |   1 +
>  drivers/media/common/videobuf2/v4l2-allocator.c    | 186 +++++++++++++++++++++
>  .../media/common/videobuf2/videobuf2-dma-contig.c  |  27 ++-
>  .../media/platform/rockchip/rkvdec/rkvdec-h264.c   |  14 +-
>  .../media/platform/rockchip/rkvdec/rkvdec-hevc.c   |  14 +-
>  .../media/platform/rockchip/rkvdec/rkvdec-rcb.c    |  21 ++-
>  .../platform/rockchip/rkvdec/rkvdec-vdpu381-h264.c |  14 +-
>  .../platform/rockchip/rkvdec/rkvdec-vdpu381-hevc.c |  14 +-
>  .../platform/rockchip/rkvdec/rkvdec-vdpu383-h264.c |  14 +-
>  .../platform/rockchip/rkvdec/rkvdec-vdpu383-hevc.c |  14 +-
>  .../media/platform/rockchip/rkvdec/rkvdec-vp9.c    |  33 ++--
>  drivers/media/platform/rockchip/rkvdec/rkvdec.c    |   4 +
>  drivers/media/platform/verisilicon/hantro.h        |   1 +
>  drivers/media/platform/verisilicon/hantro_drv.c    |   4 +
>  drivers/media/platform/verisilicon/hantro_h264.c   |   7 +-
>  drivers/media/platform/verisilicon/hantro_hevc.c   | 100 ++++++-----
>  drivers/media/platform/verisilicon/hantro_mpeg2.c  |  16 +-
>  .../media/platform/verisilicon/hantro_postproc.c   |  14 +-
>  drivers/media/platform/verisilicon/hantro_vp8.c    |  25 +--
>  drivers/media/platform/verisilicon/hantro_vp9.c    |  34 +++-
>  .../verisilicon/rockchip_vpu981_hw_av1_dec.c       | 160 +++++++++++-------
>  drivers/media/v4l2-core/v4l2-device.c              |   4 +-
>  include/media/v4l2-allocator.h                     |  26 +++
>  include/media/v4l2-device.h                        |   2 +
>  include/media/videobuf2-core.h                     |   3 +
>  25 files changed, 555 insertions(+), 197 deletions(-)
> ---
> base-commit: 66affa37cfac0aec061cc4bcf4a065b0c52f7e19
> change-id: 20260612-v4l2-add-mem-tracker-da0088c74a64
> prerequisite-change-id: 20260608-v4l2-add-ftrace-aec6e7f60a6c:v2
> prerequisite-patch-id: bd45b4df66799a7f9f22b49973a78d1bd3590a4a
> prerequisite-patch-id: 6a5ed5615f08257cf854d441785066b5ff4d5d47
> prerequisite-patch-id: 02594c869498b9e91e416d70242afbaa6b3a8d6a
> prerequisite-patch-id: 9f25ec78f3b99ac1c5d42eb732f5bd9eae73fadd
> prerequisite-patch-id: 4ed2c2e8c3abfa7f6328d0ea4f488ca5a3eec8fb
> prerequisite-patch-id: 8787d50eb80e1a63a57a123268cade23c9e64618
> prerequisite-patch-id: bfe85822378fa771f9bbc3e1e8c8ff6bce0eb7fd
> prerequisite-patch-id: 784d0801da320f7b41194f4632ee422c32f1b5b8
> prerequisite-patch-id: 362c8366e2efbfb5e6f1a3f71cfddbd46e8d1506
>
> Best regards,
> --  
> Detlev Casanova <detlev.casanova@collabora.com>
>
>
> _______________________________________________
> Linux-rockchip mailing list
> Linux-rockchip@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-rockchip