From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
To: "Du, Bin" <bin.du@amd.com>
Cc: mchehab@kernel.org, hverkuil@xs4all.nl,
bryan.odonoghue@linaro.org, sakari.ailus@linux.intel.com,
prabhakar.mahadev-lad.rj@bp.renesas.com,
linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
pratap.nirujogi@amd.com, benjamin.chan@amd.com, king.li@amd.com,
gjorgji.rosikopulos@amd.com, Phil.Jawich@amd.com,
Dominic.Antony@amd.com, Svetoslav.Stoilov@amd.com
Subject: Re: [PATCH v2 8/8] Documentation: add documentation of AMD isp 4 driver
Date: Tue, 12 Aug 2025 16:42:22 +0300 [thread overview]
Message-ID: <20250812134222.GN30054@pendragon.ideasonboard.com> (raw)
In-Reply-To: <e614b565-81e0-49c0-93dc-af1936462728@amd.com>
On Tue, Aug 12, 2025 at 09:36:04AM +0800, Du, Bin wrote:
> On 8/5/2025 7:37 PM, Laurent Pinchart wrote:
> > On Wed, Jun 18, 2025 at 05:19:59PM +0800, Bin Du wrote:
> >> Add documentation for AMD isp 4 and describe the main components
> >>
> >> Signed-off-by: Bin Du <Bin.Du@amd.com>
> >> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> >> ---
> >> Documentation/admin-guide/media/amdisp4-1.rst | 64 +++++++++++++++++++
> >> Documentation/admin-guide/media/amdisp4.dot | 8 +++
> >> MAINTAINERS | 2 +
> >> 3 files changed, 74 insertions(+)
> >> create mode 100644 Documentation/admin-guide/media/amdisp4-1.rst
> >> create mode 100644 Documentation/admin-guide/media/amdisp4.dot
> >>
> >> diff --git a/Documentation/admin-guide/media/amdisp4-1.rst b/Documentation/admin-guide/media/amdisp4-1.rst
> >> new file mode 100644
> >> index 000000000000..417b15af689a
> >> --- /dev/null
> >> +++ b/Documentation/admin-guide/media/amdisp4-1.rst
> >> @@ -0,0 +1,64 @@
> >> +.. SPDX-License-Identifier: GPL-2.0
> >> +
> >> +.. include:: <isonum.txt>
> >> +
> >> +====================================
> >> +AMD Image Signal Processor (amdisp4)
> >> +====================================
> >> +
> >> +Introduction
> >> +============
> >> +
> >> +This file documents the driver for the AMD ISP4 that is part of
> >> +AMD Ryzen AI Max 385 SoC.
> >> +
> >> +The driver is located under drivers/media/platform/amd/isp4 and uses
> >> +the Media-Controller API.
> >> +
> >> +Topology
> >> +========
> >> +
> >> +.. _amdisp4_topology_graph:
> >> +
> >> +.. kernel-figure:: amdisp4.dot
> >> + :alt: Diagram of the media pipeline topology
> >> + :align: center
> >> +
> >> +
> >> +
> >> +The driver has 1 sub-device:
> >> +
> >> +- isp: used to resize and process bayer raw frames in to yuv.
> >> +
> >> +The driver has 1 video device:
> >> +
> >> +- <capture video device: capture device for retrieving images.
> >> +
> >> +
> >> + - ISP4 Image Signal Processing Subdevice Node
> >> +-----------------------------------------------
> >> +
> >> +The isp4 is represented as a single V4L2 subdev, the sub-device does not
> >> +provide interface to the user space.
> >
> > Doesn't it ? The driver sets the V4L2_SUBDEV_FL_HAS_DEVNODE flag for the
> > subdev, and calls v4l2_device_register_subdev_nodes().
>
> We have exported subdev device to user space during the testing with
> libcamera sample pipeline.
But it's not needed anymore ? If not, you could stop exposing the subdev
to userspace for the time being.
> > As far as I understand, the camera is exposed by the firmware with a
> > webcam-like interface. We need to better understand your plans with this
> > driver. If everything is handled by the firmware, why are the sensor and
> > subdev exposed to userspace ? Why can't you expose a single video
> > capture device, with a media device, and handle everything behind the
> > scene ? I assume there may be more features coming later. Please
> > document the plan, we can't provide feedback on the architecture
> > otherwise.
>
> Currently, isp fw is controlling the sensor to update just the exposure
> and gain, since the 3A algorithms run on ISP HW rather than on x86.
This design decision makes my hair stand on end :-( Exposing the camera
sensor to both the firmware and the host concurrently is asking for
trouble. If you really want to abstract the camera behind a firmware and
only expose a webcam-like API (or not even that in this version, as the
driver exposes no control as far as I can see), then you should push the
whole sensor handling to the firmware too. In my opinion that would not
be a good solution compared to exposing the ISP to the host, but it
would be better than this hybrid model.
> In a
> future version, we plan to introduce raw output support in the ISP
> driver, allowing users to choose between AMD’s 3A running on ISP
> hardware or a custom 3A running on x86. If the user opts for the
> x86-based 3A, the firmware will relinquish control of the sensor, and
> hands over full control to the x86 system.
Will the firmware at that point expose to Linux all the ISP statistics
needed to implement auto-exposure ? What if I want to also set digital
gain ? Or have manual white balance (requiring statistics), and manual
CCM ?
> >> The sub-device is connected to one video node
> >> +(isp4_capture) with immutable active link. The isp entity is connected
> >> +to sensor pad 0 and receives the frames using CSI-2 protocol. The sub-device is
> >> +also responsible to configure CSI2-2 receiver.
> >> +The sub-device processes bayer raw data from the connected sensor and output
> >> +them to different YUV formats. The isp also has scaling capabilities.
> >> +
> >> + - isp4_capture - Frames Capture Video Node
> >> +--------------------------------------------
> >> +
> >> +Isp4_capture is a capture device to capture frames to memory.
> >> +This entity is the DMA engine that write the frames to memory.
> >> +The entity is connected to isp4 sub-device.
> >> +
> >> +Capturing Video Frames Example
> >> +==============================
> >> +
> >> +.. code-block:: bash
> >> +
> >> + # set the links
> >
> > This seems very under-documented.
>
> Yes, documentation needs to be updated.
>
> >> +
> >> + # start streaming:
> >> + v4l2-ctl "-d" "/dev/video0" "--set-fmt-video=width=1920,height=1080,pixelformat=NV12" "--stream-mmap" "--stream-count=10"
> >> diff --git a/Documentation/admin-guide/media/amdisp4.dot b/Documentation/admin-guide/media/amdisp4.dot
> >> new file mode 100644
> >> index 000000000000..a4c2f0cceb30
> >> --- /dev/null
> >> +++ b/Documentation/admin-guide/media/amdisp4.dot
> >> @@ -0,0 +1,8 @@
> >> +digraph board {
> >> + rankdir=TB
> >> + n00000001 [label="{{<port0> 0} | amd isp4\n | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> >> + n00000001:port1 -> n00000004 [style=bold]
> >> + n00000004 [label="Preview\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> >> + n0000000a [label="{{} | ov05c10 22-0010\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> >> + n0000000a:port0 -> n00000001:port0 [style=bold]
> >> +}
> >> diff --git a/MAINTAINERS b/MAINTAINERS
> >> index 15070afb14b5..e4455bde376f 100644
> >> --- a/MAINTAINERS
> >> +++ b/MAINTAINERS
> >> @@ -1113,6 +1113,8 @@ M: Nirujogi Pratap <pratap.nirujogi@amd.com>
> >> L: linux-media@vger.kernel.org
> >> S: Maintained
> >> T: git git://linuxtv.org/media.git
> >> +F: Documentation/admin-guide/media/amdisp4-1.rst
> >> +F: Documentation/admin-guide/media/amdisp4.dot
> >> F: drivers/media/platform/amd/Kconfig
> >> F: drivers/media/platform/amd/Makefile
> >> F: drivers/media/platform/amd/isp4/*
--
Regards,
Laurent Pinchart
next prev parent reply other threads:[~2025-08-12 13:42 UTC|newest]
Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-18 9:19 [PATCH v2 0/8] Add AMD ISP4 driver Bin Du
2025-06-18 9:19 ` [PATCH v2 1/8] media: platform: amd: Introduce amd isp4 capture driver Bin Du
2025-06-18 15:58 ` Mario Limonciello
2025-06-19 7:46 ` Du, Bin
2025-06-19 13:00 ` Mario Limonciello
2025-06-20 3:08 ` Du, Bin
2025-07-28 5:54 ` Sakari Ailus
2025-07-28 9:00 ` Du, Bin
2025-06-18 9:19 ` [PATCH v2 2/8] media: platform: amd: low level support for isp4 firmware Bin Du
2025-06-18 16:00 ` Mario Limonciello
2025-06-19 7:53 ` Du, Bin
2025-07-28 5:57 ` Sakari Ailus
2025-07-28 9:24 ` Du, Bin
2025-06-18 9:19 ` [PATCH v2 3/8] media: platform: amd: Add helpers to configure isp4 mipi phy Bin Du
2025-07-28 6:33 ` Sakari Ailus
2025-08-05 9:53 ` Du, Bin
2025-08-05 10:53 ` Laurent Pinchart
2025-08-06 9:56 ` Du, Bin
2025-08-05 10:39 ` Laurent Pinchart
2025-08-06 9:45 ` Du, Bin
2025-07-28 7:28 ` Sakari Ailus
2025-07-31 9:31 ` Du, Bin
2025-06-18 9:19 ` [PATCH v2 4/8] media: platform: amd: Add isp4 fw and hw interface Bin Du
2025-06-18 16:17 ` Mario Limonciello
2025-06-19 9:58 ` Du, Bin
2025-06-19 15:11 ` Mario Limonciello
2025-06-20 3:32 ` Du, Bin
2025-07-28 7:23 ` Sakari Ailus
2025-07-29 9:12 ` Du, Bin
2025-08-11 11:46 ` Sakari Ailus
2025-08-11 12:31 ` Laurent Pinchart
2025-08-12 3:36 ` Du, Bin
2025-08-12 7:34 ` Laurent Pinchart
2025-08-12 8:08 ` Du, Bin
2025-08-12 8:20 ` Sakari Ailus
2025-08-12 10:04 ` Du, Bin
2025-08-12 2:44 ` Du, Bin
2025-06-18 9:19 ` [PATCH v2 5/8] media: platform: amd: isp4 subdev and firmware loading handling added Bin Du
2025-06-18 16:35 ` Mario Limonciello
2025-06-20 9:31 ` Du, Bin
2025-07-06 20:55 ` Mario Limonciello
2025-07-07 6:22 ` Du, Bin
2025-07-25 1:35 ` Sultan Alsawaf
2025-07-25 9:03 ` Du, Bin
2025-06-18 9:19 ` [PATCH v2 6/8] media: platform: amd: isp4 video node and buffers " Bin Du
2025-07-23 17:55 ` Sultan Alsawaf
2025-07-24 5:14 ` Sultan Alsawaf
2025-07-25 9:05 ` Du, Bin
2025-07-25 9:22 ` Du, Bin
2025-07-26 21:41 ` Sultan Alsawaf
2025-07-26 21:50 ` Sultan Alsawaf
2025-07-29 6:12 ` Du, Bin
2025-07-29 6:08 ` Du, Bin
2025-07-28 7:04 ` Sultan Alsawaf
2025-07-29 7:43 ` Du, Bin
2025-07-31 0:34 ` Sultan Alsawaf
2025-07-31 9:45 ` Du, Bin
2025-08-11 6:02 ` Sultan Alsawaf
2025-08-11 9:05 ` Du, Bin
2025-08-12 5:51 ` Sultan Alsawaf
2025-08-12 6:33 ` Du, Bin
2025-08-13 9:42 ` Du, Bin
2025-08-14 6:37 ` Sultan Alsawaf
2025-06-18 9:19 ` [PATCH v2 7/8] media: platform: amd: isp4 debug fs logging and more descriptive errors Bin Du
2025-06-18 9:19 ` [PATCH v2 8/8] Documentation: add documentation of AMD isp 4 driver Bin Du
2025-08-05 11:37 ` Laurent Pinchart
2025-08-12 1:36 ` Du, Bin
2025-08-12 13:42 ` Laurent Pinchart [this message]
2025-08-22 2:28 ` Du, Bin
2025-08-20 12:42 ` Sakari Ailus
2025-08-22 2:20 ` Du, Bin
2025-09-22 6:24 ` Sakari Ailus
2025-09-22 9:19 ` Du, Bin
2025-07-23 18:12 ` [PATCH v2 0/8] Add AMD ISP4 driver Sultan Alsawaf
2025-07-25 10:22 ` Du, Bin
2025-07-26 22:31 ` Sultan Alsawaf
2025-07-29 3:32 ` Du, Bin
2025-07-29 7:42 ` Sultan Alsawaf
2025-07-29 7:45 ` Sultan Alsawaf
2025-07-29 10:13 ` Du, Bin
2025-07-30 5:38 ` Sultan Alsawaf
2025-07-30 9:53 ` Du, Bin
2025-07-31 0:30 ` Sultan Alsawaf
2025-07-31 10:04 ` Du, Bin
2025-08-04 3:32 ` Du, Bin
2025-08-04 4:25 ` Sultan Alsawaf
2025-08-08 9:11 ` Du, Bin
2025-08-11 5:49 ` Sultan Alsawaf
2025-08-11 8:35 ` Du, Bin
2025-08-11 21:48 ` Sultan Alsawaf
2025-08-11 22:17 ` Sultan Alsawaf
2025-08-12 2:02 ` Du, Bin
2025-08-14 6:53 ` Sultan Alsawaf
2025-08-22 2:23 ` Du, Bin
2025-08-22 3:56 ` Sultan Alsawaf
2025-08-27 10:30 ` Du, Bin
2025-08-28 5:50 ` Sultan Alsawaf
2025-09-02 2:08 ` Du, Bin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250812134222.GN30054@pendragon.ideasonboard.com \
--to=laurent.pinchart@ideasonboard.com \
--cc=Dominic.Antony@amd.com \
--cc=Phil.Jawich@amd.com \
--cc=Svetoslav.Stoilov@amd.com \
--cc=benjamin.chan@amd.com \
--cc=bin.du@amd.com \
--cc=bryan.odonoghue@linaro.org \
--cc=gjorgji.rosikopulos@amd.com \
--cc=hverkuil@xs4all.nl \
--cc=king.li@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=prabhakar.mahadev-lad.rj@bp.renesas.com \
--cc=pratap.nirujogi@amd.com \
--cc=sakari.ailus@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®