Re: [PATCH v1] media: videobuf2: Allow applications customize data offsets of capture buffers

[Hans Verkuil <hverkuil+cisco@kernel.org>]

On 15/12/2025 21:56, Nicolas Dufresne wrote:
> Hi Hirokazu, Hans,
> 
> Le vendredi 12 décembre 2025 à 15:15 -0800, Hirokazu Honda a écrit :
>> Thanks Hans for your quick response.
>> And I apologize for my late reply.
>>
>>> So that's were I am: I'm not opposed to supporting this feature, but I
>>> feel that struct v4l2_buffer has reached end-of-life, and that it is
>>> time for a modern replacement.
>>
>> I got it.
>> I will discuss in my team if I can contribute to v4l2_buffer_ext work
>> or sponsor that work next year or in 2027.
> 
> I share Hans feeling in regard to trying to force partial support for
> data_offset into some space left of v4l2_buffer. This brought me thinking that
> doing that, or even v4l2_buffer_ext as last proposed was not the right solution
> to modernize the old V4L2 framework. In fact, I believe that v4l2_buffer_ext
> would simply replicate the MPLANE disaster, leaving another permanent scar in
> the API. Just to state simply, MPLANE have lead to duplication of every multi-
> plane pixel formats, solving some plane alignment issue in multi-allocation
> cases, but leaving behind the common single allocation case.
> 
> For me, the most central issue in V4L2 is that the memory allocation/importation
> is bound to the operation queues. That brings all sort of issues such 
> 
> - We can't queue twice the same frame
> - We can't mix external buffer with device allocated buffer
> - All buffers must have the exact same stride

The three limitations above are all technically possible to implement with the
current vb2 framework/streaming uAPI, it's just that nobody was ever motivated
enough to add support for it.

> - Application is responsible for caching which memory goes to which v4l2_buffer

True, but is this really a big deal?

> - Attempting to import a buffer requires a free spot in the queue

True.

> 
> This adds on top of the v4l2_buffer structure limitation we have been targeting
> so far. With the growth of modern standard API (think Vulkan Video notably), it
> becomes apparent that the model is too inflexible. This inability to separate
> memory allocation and importation from operations creates a lot of complexity in
> user-space, leading to complicated bugs.
> 
> I've been quite about it, since until now I didn't have a solution in mind, but
> I recently come with some ideas. I'll will try develop these ideas, at least in
> prose for now and come up with an RFC, hopefully somewhere beginning of January
> 2026. That my proposal is accepted or not isn't quite relevant. But hopefully it
> will be a starter to go go beyond just fixing what we see. In fact, this next
> step is for me doing to be quite decisive if I continue doing codecs in V4L2 or
> not in the long run. But I'm sure this is not just about video codecs.

I'm looking forward to your RFC!

What is important for me is that whatever we come up with, it is something that
existing drivers can easily support. A new streaming/buffer allocation uAPI that
can only be supported by new drivers will need very, very good reasons for it to
be accepted.

The nice thing about the v4l2_buffer_ext proposal was that it can easily be
supported by all drivers.

From a technical perspective both struct v4l2_buffer and struct v4l2_format have
reached end-of-life: they are full of historical cruft, they are inefficient, the
32-bit/time32 compatibility code is awful and hard to maintain.

Regards,

	Hans

> 
> regards,
> Nicolas