mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH 0/2] media: atomisp: Fix PipeWire/Firefox streaming for sensors without crop or frame interval support
@ 2026-11-16 17:57 Volodymyr Chapman
  2026-11-16 17:57 ` [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output Volodymyr Chapman
  2026-11-16 17:57 ` [PATCH 2/2] media: atomisp: Report frame interval for sensors without enum_frame_interval Volodymyr Chapman
  0 siblings, 2 replies; 4+ messages in thread
From: Volodymyr Chapman @ 2026-11-16 17:57 UTC (permalink / raw)
  To: Hans de Goede, Mauro Carvalho Chehab
  Cc: Sakari Ailus, Andy Shevchenko, Greg Kroah-Hartman, linux-media,
	linux-staging, linux-kernel, Volodymyr Chapman

With PipeWire as the camera path, Firefox could not stream from the two
cameras of a Lenovo ThinkPad 10 (20C3) on atomisp:

- Patch 1: for sensors without crop support, VIDIOC_ENUM_FRAMESIZES lists
  sizes which the preview pipeline cannot output, so format negotiation
  fails when userspace picks the largest one (IMX175 3264x2448).
- Patch 2: VIDIOC_ENUM_FRAMEINTERVALS fails for sensors without
  enum_frame_interval, so PipeWire advertises a 1-120 fps range, which
  Firefox turns into 1 fps. atomisp now reports the sensor's current
  frame interval from get_frame_interval or the pixel rate and blanking
  controls.

Patch 2 has no Fixes tag: ENUM_FRAMEINTERVALS never worked for these
sensors, so this is more an improvement than a regression fix.

Tested on the ThinkPad 10 with the OV2722 front sensor and the IMX175 rear
sensor (the IMX175 driver is pending in a separate series [1]), with both
patches built out of tree against a 7.2.15 kernel: Firefox 160 streams from
both sensors at about 30 fps through PipeWire. Build-tested on media.git
next.

[1] https://lore.kernel.org/linux-media/20261116175151.20150-1-volodymyrchapman@gmail.com/

The problems were investigated and the patches drafted with the help of an
AI coding assistant (Claude); the testing was done on the hardware by the
submitter.

Volodymyr Chapman (2):
  media: atomisp: Don't enumerate frame sizes the preview pipe can't
    output
  media: atomisp: Report frame interval for sensors without
    enum_frame_interval

 .../staging/media/atomisp/pci/atomisp_ioctl.c | 86 ++++++++++++++++++-
 1 file changed, 83 insertions(+), 3 deletions(-)


base-commit: 3fc9303dba01676c2090768126e5327e341b7761
-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output
  2026-11-16 17:57 [PATCH 0/2] media: atomisp: Fix PipeWire/Firefox streaming for sensors without crop or frame interval support Volodymyr Chapman
@ 2026-11-16 17:57 ` Volodymyr Chapman
  2026-11-17 10:31   ` Andy Shevchenko
  2026-11-16 17:57 ` [PATCH 2/2] media: atomisp: Report frame interval for sensors without enum_frame_interval Volodymyr Chapman
  1 sibling, 1 reply; 4+ messages in thread
From: Volodymyr Chapman @ 2026-11-16 17:57 UTC (permalink / raw)
  To: Hans de Goede, Mauro Carvalho Chehab
  Cc: Sakari Ailus, Andy Shevchenko, Greg Kroah-Hartman, linux-media,
	linux-staging, linux-kernel, Volodymyr Chapman

In preview mode atomisp_try_fmt() limits the size to 1920x1440, since the
preview pipeline cannot output anything larger. For sensors without crop
support atomisp_enum_framesizes() however passes on all sensor sizes,
minus the padding, so VIDIOC_ENUM_FRAMESIZES lists sizes for which
VIDIOC_S_FMT returns a different, smaller size.

Userspace which picks the largest enumerated size and then checks that
S_FMT gave it that size fails. For example PipeWire's V4L2 source cannot
negotiate a format with Firefox for the IMX175 rear sensor of a Lenovo
ThinkPad 10, which lists 3264x2448 and 1624x1216: Firefox picks
3264x2448, S_FMT returns 1624x1216 and the stream fails with "error set
output format: -22 (Invalid argument)".

Skip the sizes which do not fit the preview pipeline when in preview mode,
so that only sizes which S_FMT can set are enumerated.

Tested on a Lenovo ThinkPad 10 (20C3, Atom Z3795) with this file built out
of tree against a 7.2.15 kernel: ENUM_FRAMESIZES for the IMX175 now lists
only 1624x1216, and Firefox 160 streams from both the IMX175 and the
OV2722 front sensor through PipeWire. This was build-tested on top of
media.git next.

The problem was investigated and this patch drafted with the help of an
AI coding assistant (Claude); the testing was done on the hardware by the
submitter.

Fixes: d86c33af672d ("media: atomisp: Clamp width to max 1920 pixels when in ATOMISP_RUN_MODE_PREVIEW")
Signed-off-by: Volodymyr Chapman <volodymyrchapman@gmail.com>
---
 .../staging/media/atomisp/pci/atomisp_ioctl.c | 22 ++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
index 50366bf10..046a95857 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
@@ -532,11 +532,11 @@ static int atomisp_enum_framesizes(struct file *file, void *priv,
 	struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd;
 	struct atomisp_input_subdev *input = &isp->inputs[asd->input_curr];
 	struct v4l2_subdev_frame_size_enum fse = {
-		.index = fsize->index,
 		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
 		.code = input->code,
 	};
 	struct v4l2_subdev_state *act_sd_state;
+	unsigned int valid_sizes = 0;
 	int ret;
 
 	if (!input->sensor)
@@ -546,8 +546,24 @@ static int atomisp_enum_framesizes(struct file *file, void *priv,
 		return atomisp_enum_framesizes_crop(isp, fsize);
 
 	act_sd_state = v4l2_subdev_lock_and_get_active_state(input->sensor);
-	ret = v4l2_subdev_call(input->sensor, pad, enum_frame_size,
-			       act_sd_state, &fse);
+	for (fse.index = 0; ; fse.index++) {
+		ret = v4l2_subdev_call(input->sensor, pad, enum_frame_size,
+				       act_sd_state, &fse);
+		if (ret)
+			break;
+
+		/*
+		 * Skip sizes the preview pipeline cannot output, for which
+		 * atomisp_try_fmt() would return a smaller size.
+		 */
+		if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW &&
+		    (fse.max_width - pad_w > 1920 ||
+		     fse.max_height - pad_h > 1440))
+			continue;
+
+		if (valid_sizes++ == fsize->index)
+			break;
+	}
 	if (act_sd_state)
 		v4l2_subdev_unlock_state(act_sd_state);
 	if (ret)
-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 2/2] media: atomisp: Report frame interval for sensors without enum_frame_interval
  2026-11-16 17:57 [PATCH 0/2] media: atomisp: Fix PipeWire/Firefox streaming for sensors without crop or frame interval support Volodymyr Chapman
  2026-11-16 17:57 ` [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output Volodymyr Chapman
@ 2026-11-16 17:57 ` Volodymyr Chapman
  1 sibling, 0 replies; 4+ messages in thread
From: Volodymyr Chapman @ 2026-11-16 17:57 UTC (permalink / raw)
  To: Hans de Goede, Mauro Carvalho Chehab
  Cc: Sakari Ailus, Andy Shevchenko, Greg Kroah-Hartman, linux-media,
	linux-staging, linux-kernel, Volodymyr Chapman

atomisp_enum_frameintervals() passes VIDIOC_ENUM_FRAMEINTERVALS on to the
sensor's enum_frame_interval pad op. Most sensor drivers used with atomisp
do not implement it, e.g. the OV2722 driver only implements
get_frame_interval and raw sensor drivers such as the IMX175 driver set
their frame rate with the pixel rate and blanking controls instead. For
these sensors VIDIOC_ENUM_FRAMEINTERVALS fails, so userspace does not know
the frame rate.

PipeWire's V4L2 source then advertises a framerate range of 1/1 to 120/1.
Firefox (libwebrtc) takes the minimum of such a range as the camera's
maximum framerate and drops all but 1 frame per second, while the sensor
runs at 30 fps.

When the sensor has no enum_frame_interval, report one discrete frame
interval: the sensor's get_frame_interval if it has that, or else the one
following from its V4L2_CID_PIXEL_RATE, V4L2_CID_HBLANK and V4L2_CID_VBLANK
controls and active format, rounded to a whole number of frames per
second. This is the interval of the sensor's current mode, which is
reported for every frame size; for the OV2722 and IMX175 all modes run at
30 fps.

Tested on a Lenovo ThinkPad 10 (20C3, Atom Z3795) with this file built out
of tree against a 7.2.15 kernel: ENUM_FRAMEINTERVALS reports 30 fps for
both the OV2722 front sensor (from get_frame_interval) and the IMX175 rear
sensor (256 MHz / (3430 * 2500)), and webcamtests.com in Firefox 160 shows
about 30 fps for both instead of 1 fps. This was build-tested on top of
media.git next.

The problem was investigated and this patch drafted with the help of an
AI coding assistant (Claude); the testing was done on the hardware by the
submitter.

Signed-off-by: Volodymyr Chapman <volodymyrchapman@gmail.com>
---
 .../staging/media/atomisp/pci/atomisp_ioctl.c | 64 +++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
index 046a95857..c83834405 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
@@ -576,6 +576,65 @@ static int atomisp_enum_framesizes(struct file *file, void *priv,
 	return 0;
 }
 
+/*
+ * For sensors without enum_frame_interval, get the frame interval of the
+ * sensor's current mode: from get_frame_interval, or else from the pixel rate
+ * and blanking controls which raw sensor drivers use to set the frame rate.
+ */
+static int atomisp_sensor_frame_interval(struct atomisp_input_subdev *input,
+					 struct v4l2_fract *interval)
+{
+	struct v4l2_subdev_frame_interval fi = {
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+	};
+	struct v4l2_subdev_format fmt = {
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+	};
+	struct v4l2_ctrl *pixel_rate, *hblank, *vblank;
+	struct v4l2_subdev_state *act_sd_state;
+	u64 frame_size, fps;
+	int ret;
+
+	act_sd_state = v4l2_subdev_lock_and_get_active_state(input->sensor);
+	ret = v4l2_subdev_call(input->sensor, pad, get_frame_interval,
+			       act_sd_state, &fi);
+	if (ret != -ENOIOCTLCMD) {
+		if (act_sd_state)
+			v4l2_subdev_unlock_state(act_sd_state);
+		if (ret)
+			return ret;
+		if (!fi.interval.numerator || !fi.interval.denominator)
+			return -EINVAL;
+		*interval = fi.interval;
+		return 0;
+	}
+
+	ret = v4l2_subdev_call(input->sensor, pad, get_fmt, act_sd_state, &fmt);
+	if (act_sd_state)
+		v4l2_subdev_unlock_state(act_sd_state);
+	if (ret)
+		return ret;
+
+	pixel_rate = v4l2_ctrl_find(input->sensor->ctrl_handler,
+				    V4L2_CID_PIXEL_RATE);
+	hblank = v4l2_ctrl_find(input->sensor->ctrl_handler, V4L2_CID_HBLANK);
+	vblank = v4l2_ctrl_find(input->sensor->ctrl_handler, V4L2_CID_VBLANK);
+	if (!pixel_rate || !hblank || !vblank)
+		return -EINVAL;
+
+	frame_size = (u64)(fmt.format.width + v4l2_ctrl_g_ctrl(hblank)) *
+		     (fmt.format.height + v4l2_ctrl_g_ctrl(vblank));
+	fps = div64_u64(v4l2_ctrl_g_ctrl_int64(pixel_rate) + frame_size / 2,
+			frame_size);
+	if (!fps)
+		return -EINVAL;
+
+	interval->numerator = 1;
+	interval->denominator = fps;
+
+	return 0;
+}
+
 static int atomisp_enum_frameintervals(struct file *file, void *priv,
 				       struct v4l2_frmivalenum *fival)
 {
@@ -601,6 +660,11 @@ static int atomisp_enum_frameintervals(struct file *file, void *priv,
 			       act_sd_state, &fie);
 	if (act_sd_state)
 		v4l2_subdev_unlock_state(act_sd_state);
+	if (ret == -ENOIOCTLCMD) {
+		if (fival->index)
+			return -EINVAL;
+		ret = atomisp_sensor_frame_interval(input, &fie.interval);
+	}
 	if (ret)
 		return ret;
 
-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output
  2026-11-16 17:57 ` [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output Volodymyr Chapman
@ 2026-11-17 10:31   ` Andy Shevchenko
  0 siblings, 0 replies; 4+ messages in thread
From: Andy Shevchenko @ 2026-11-17 10:31 UTC (permalink / raw)
  To: Volodymyr Chapman
  Cc: Hans de Goede, Mauro Carvalho Chehab, Sakari Ailus,
	Andy Shevchenko, Greg Kroah-Hartman, linux-media, linux-staging,
	linux-kernel

On Mon, Nov 16, 2026 at 05:57:49PM +0000, Volodymyr Chapman wrote:
> In preview mode atomisp_try_fmt() limits the size to 1920x1440, since the
> preview pipeline cannot output anything larger. For sensors without crop support

So, what makes us stop implementing the crop support there?

> atomisp_enum_framesizes() however passes on all sensor sizes,
> minus the padding, so VIDIOC_ENUM_FRAMESIZES lists sizes for which
> VIDIOC_S_FMT returns a different, smaller size.
> 
> Userspace which picks the largest enumerated size and then checks that
> S_FMT gave it that size fails. For example PipeWire's V4L2 source cannot
> negotiate a format with Firefox for the IMX175 rear sensor of a Lenovo
> ThinkPad 10, which lists 3264x2448 and 1624x1216: Firefox picks
> 3264x2448, S_FMT returns 1624x1216 and the stream fails with "error set
> output format: -22 (Invalid argument)".
> 
> Skip the sizes which do not fit the preview pipeline when in preview mode,
> so that only sizes which S_FMT can set are enumerated.

> Tested on a Lenovo ThinkPad 10 (20C3, Atom Z3795) with this file built out
> of tree against a 7.2.15 kernel: ENUM_FRAMESIZES for the IMX175 now lists
> only 1624x1216, and Firefox 160 streams from both the IMX175 and the
> OV2722 front sensor through PipeWire. This was build-tested on top of
> media.git next.
> 
> The problem was investigated and this patch drafted with the help of an
> AI coding assistant (Claude); the testing was done on the hardware by the
> submitter.

No need to repeat these two paragraphs, make sure they are in the cover letter.

...

> -	ret = v4l2_subdev_call(input->sensor, pad, enum_frame_size,
> -			       act_sd_state, &fse);
> +	for (fse.index = 0; ; fse.index++) {
> +		ret = v4l2_subdev_call(input->sensor, pad, enum_frame_size,
> +				       act_sd_state, &fse);
> +		if (ret)
> +			break;
> +
> +		/*
> +		 * Skip sizes the preview pipeline cannot output, for which
> +		 * atomisp_try_fmt() would return a smaller size.
> +		 */
> +		if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW &&
> +		    (fse.max_width - pad_w > 1920 ||
> +		     fse.max_height - pad_h > 1440))
> +			continue;
> +
> +		if (valid_sizes++ == fsize->index)
> +			break;
> +	}

TBH, this feels like a hack, but I'm not v4l2/media reviewer/maintainer,
so I don't know if this approach is legit.

-- 
With Best Regards,
Andy Shevchenko



^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-11-17 10:31 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-16 17:57 [PATCH 0/2] media: atomisp: Fix PipeWire/Firefox streaming for sensors without crop or frame interval support Volodymyr Chapman
2026-11-16 17:57 ` [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output Volodymyr Chapman
2026-11-17 10:31   ` Andy Shevchenko
2026-11-16 17:57 ` [PATCH 2/2] media: atomisp: Report frame interval for sensors without enum_frame_interval Volodymyr Chapman

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®