* [PATCH 0/2] media: atomisp: Fix PipeWire/Firefox streaming for sensors without crop or frame interval support
@ 2026-11-16 17:57 Volodymyr Chapman
2026-11-16 17:57 ` [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output Volodymyr Chapman
2026-11-16 17:57 ` [PATCH 2/2] media: atomisp: Report frame interval for sensors without enum_frame_interval Volodymyr Chapman
0 siblings, 2 replies; 6+ messages in thread
From: Volodymyr Chapman @ 2026-11-16 17:57 UTC (permalink / raw)
To: Hans de Goede, Mauro Carvalho Chehab
Cc: Sakari Ailus, Andy Shevchenko, Greg Kroah-Hartman, linux-media,
linux-staging, linux-kernel, Volodymyr Chapman
With PipeWire as the camera path, Firefox could not stream from the two
cameras of a Lenovo ThinkPad 10 (20C3) on atomisp:
- Patch 1: for sensors without crop support, VIDIOC_ENUM_FRAMESIZES lists
sizes which the preview pipeline cannot output, so format negotiation
fails when userspace picks the largest one (IMX175 3264x2448).
- Patch 2: VIDIOC_ENUM_FRAMEINTERVALS fails for sensors without
enum_frame_interval, so PipeWire advertises a 1-120 fps range, which
Firefox turns into 1 fps. atomisp now reports the sensor's current
frame interval from get_frame_interval or the pixel rate and blanking
controls.
Patch 2 has no Fixes tag: ENUM_FRAMEINTERVALS never worked for these
sensors, so this is more an improvement than a regression fix.
Tested on the ThinkPad 10 with the OV2722 front sensor and the IMX175 rear
sensor (the IMX175 driver is pending in a separate series [1]), with both
patches built out of tree against a 7.2.15 kernel: Firefox 160 streams from
both sensors at about 30 fps through PipeWire. Build-tested on media.git
next.
[1] https://lore.kernel.org/linux-media/20261116175151.20150-1-volodymyrchapman@gmail.com/
The problems were investigated and the patches drafted with the help of an
AI coding assistant (Claude); the testing was done on the hardware by the
submitter.
Volodymyr Chapman (2):
media: atomisp: Don't enumerate frame sizes the preview pipe can't
output
media: atomisp: Report frame interval for sensors without
enum_frame_interval
.../staging/media/atomisp/pci/atomisp_ioctl.c | 86 ++++++++++++++++++-
1 file changed, 83 insertions(+), 3 deletions(-)
base-commit: 3fc9303dba01676c2090768126e5327e341b7761
--
2.43.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output 2026-11-16 17:57 [PATCH 0/2] media: atomisp: Fix PipeWire/Firefox streaming for sensors without crop or frame interval support Volodymyr Chapman @ 2026-11-16 17:57 ` Volodymyr Chapman 2026-11-17 10:31 ` Andy Shevchenko 2026-11-16 17:57 ` [PATCH 2/2] media: atomisp: Report frame interval for sensors without enum_frame_interval Volodymyr Chapman 1 sibling, 1 reply; 6+ messages in thread From: Volodymyr Chapman @ 2026-11-16 17:57 UTC (permalink / raw) To: Hans de Goede, Mauro Carvalho Chehab Cc: Sakari Ailus, Andy Shevchenko, Greg Kroah-Hartman, linux-media, linux-staging, linux-kernel, Volodymyr Chapman In preview mode atomisp_try_fmt() limits the size to 1920x1440, since the preview pipeline cannot output anything larger. For sensors without crop support atomisp_enum_framesizes() however passes on all sensor sizes, minus the padding, so VIDIOC_ENUM_FRAMESIZES lists sizes for which VIDIOC_S_FMT returns a different, smaller size. Userspace which picks the largest enumerated size and then checks that S_FMT gave it that size fails. For example PipeWire's V4L2 source cannot negotiate a format with Firefox for the IMX175 rear sensor of a Lenovo ThinkPad 10, which lists 3264x2448 and 1624x1216: Firefox picks 3264x2448, S_FMT returns 1624x1216 and the stream fails with "error set output format: -22 (Invalid argument)". Skip the sizes which do not fit the preview pipeline when in preview mode, so that only sizes which S_FMT can set are enumerated. Tested on a Lenovo ThinkPad 10 (20C3, Atom Z3795) with this file built out of tree against a 7.2.15 kernel: ENUM_FRAMESIZES for the IMX175 now lists only 1624x1216, and Firefox 160 streams from both the IMX175 and the OV2722 front sensor through PipeWire. This was build-tested on top of media.git next. The problem was investigated and this patch drafted with the help of an AI coding assistant (Claude); the testing was done on the hardware by the submitter. Fixes: d86c33af672d ("media: atomisp: Clamp width to max 1920 pixels when in ATOMISP_RUN_MODE_PREVIEW") Signed-off-by: Volodymyr Chapman <volodymyrchapman@gmail.com> --- .../staging/media/atomisp/pci/atomisp_ioctl.c | 22 ++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c index 50366bf10..046a95857 100644 --- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c +++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c @@ -532,11 +532,11 @@ static int atomisp_enum_framesizes(struct file *file, void *priv, struct atomisp_sub_device *asd = atomisp_to_video_pipe(vdev)->asd; struct atomisp_input_subdev *input = &isp->inputs[asd->input_curr]; struct v4l2_subdev_frame_size_enum fse = { - .index = fsize->index, .which = V4L2_SUBDEV_FORMAT_ACTIVE, .code = input->code, }; struct v4l2_subdev_state *act_sd_state; + unsigned int valid_sizes = 0; int ret; if (!input->sensor) @@ -546,8 +546,24 @@ static int atomisp_enum_framesizes(struct file *file, void *priv, return atomisp_enum_framesizes_crop(isp, fsize); act_sd_state = v4l2_subdev_lock_and_get_active_state(input->sensor); - ret = v4l2_subdev_call(input->sensor, pad, enum_frame_size, - act_sd_state, &fse); + for (fse.index = 0; ; fse.index++) { + ret = v4l2_subdev_call(input->sensor, pad, enum_frame_size, + act_sd_state, &fse); + if (ret) + break; + + /* + * Skip sizes the preview pipeline cannot output, for which + * atomisp_try_fmt() would return a smaller size. + */ + if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW && + (fse.max_width - pad_w > 1920 || + fse.max_height - pad_h > 1440)) + continue; + + if (valid_sizes++ == fsize->index) + break; + } if (act_sd_state) v4l2_subdev_unlock_state(act_sd_state); if (ret) -- 2.43.0 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output 2026-11-16 17:57 ` [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output Volodymyr Chapman @ 2026-11-17 10:31 ` Andy Shevchenko 2026-11-17 17:56 ` Volodymyr Chapman 0 siblings, 1 reply; 6+ messages in thread From: Andy Shevchenko @ 2026-11-17 10:31 UTC (permalink / raw) To: Volodymyr Chapman Cc: Hans de Goede, Mauro Carvalho Chehab, Sakari Ailus, Andy Shevchenko, Greg Kroah-Hartman, linux-media, linux-staging, linux-kernel On Mon, Nov 16, 2026 at 05:57:49PM +0000, Volodymyr Chapman wrote: > In preview mode atomisp_try_fmt() limits the size to 1920x1440, since the > preview pipeline cannot output anything larger. For sensors without crop support So, what makes us stop implementing the crop support there? > atomisp_enum_framesizes() however passes on all sensor sizes, > minus the padding, so VIDIOC_ENUM_FRAMESIZES lists sizes for which > VIDIOC_S_FMT returns a different, smaller size. > > Userspace which picks the largest enumerated size and then checks that > S_FMT gave it that size fails. For example PipeWire's V4L2 source cannot > negotiate a format with Firefox for the IMX175 rear sensor of a Lenovo > ThinkPad 10, which lists 3264x2448 and 1624x1216: Firefox picks > 3264x2448, S_FMT returns 1624x1216 and the stream fails with "error set > output format: -22 (Invalid argument)". > > Skip the sizes which do not fit the preview pipeline when in preview mode, > so that only sizes which S_FMT can set are enumerated. > Tested on a Lenovo ThinkPad 10 (20C3, Atom Z3795) with this file built out > of tree against a 7.2.15 kernel: ENUM_FRAMESIZES for the IMX175 now lists > only 1624x1216, and Firefox 160 streams from both the IMX175 and the > OV2722 front sensor through PipeWire. This was build-tested on top of > media.git next. > > The problem was investigated and this patch drafted with the help of an > AI coding assistant (Claude); the testing was done on the hardware by the > submitter. No need to repeat these two paragraphs, make sure they are in the cover letter. ... > - ret = v4l2_subdev_call(input->sensor, pad, enum_frame_size, > - act_sd_state, &fse); > + for (fse.index = 0; ; fse.index++) { > + ret = v4l2_subdev_call(input->sensor, pad, enum_frame_size, > + act_sd_state, &fse); > + if (ret) > + break; > + > + /* > + * Skip sizes the preview pipeline cannot output, for which > + * atomisp_try_fmt() would return a smaller size. > + */ > + if (asd->run_mode->val == ATOMISP_RUN_MODE_PREVIEW && > + (fse.max_width - pad_w > 1920 || > + fse.max_height - pad_h > 1440)) > + continue; > + > + if (valid_sizes++ == fsize->index) > + break; > + } TBH, this feels like a hack, but I'm not v4l2/media reviewer/maintainer, so I don't know if this approach is legit. -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output 2026-11-17 10:31 ` Andy Shevchenko @ 2026-11-17 17:56 ` Volodymyr Chapman 2026-11-18 9:51 ` Andy Shevchenko 0 siblings, 1 reply; 6+ messages in thread From: Volodymyr Chapman @ 2026-11-17 17:56 UTC (permalink / raw) To: andriy.shevchenko Cc: hansg, mchehab, sakari.ailus, andy, gregkh, linux-media, linux-staging, linux-kernel Hi Andy, On Tue, Nov 17, 2026 at 12:31:31PM +0200, Andy Shevchenko wrote: > So, what makes us stop implementing the crop support there? atomisp only uses its crop path for sensors whose driver supports the NATIVE_SIZE, CROP_DEFAULT and CROP selection targets (atomisp_init_sensor()). The IMX175 driver does not yet: in his review of it, Dave Stevenson suggested that binning, and possibly cropping, should wait for the Common Raw Sensor Model. Once the sensor supports cropping, atomisp takes the atomisp_enum_framesizes_crop() path and this code is not used for it. Even with crop support, preview mode would list nothing larger than 1920x1440: that is the largest entry in the frame_sizes table of atomisp_enum_framesizes_crop_inner(). This patch applies the same limit to the non-crop path. > No need to repeat these two paragraphs, make sure they are in the cover letter. Of course. I will keep them in the cover letter only in v2. > TBH, this feels like a hack, but I'm not v4l2/media reviewer/maintainer, > so I don't know if this approach is legit. The intent is to make VIDIOC_ENUM_FRAMESIZES match what atomisp_try_fmt() returns, which already clamps the size to 1920x1440 in preview mode. Without this patch the IMX175 lists 3264x2448 and 1624x1216, and TRY_FMT for 3264x2448 returns 1624x1216. With it, the IMX175 lists only 1624x1216 and the OV2722 lists 1916x1076, 1616x1076 and 1436x1076. Every listed size is returned unchanged by TRY_FMT, and for the other sizes I tried (640x480 up to 3264x2448) TRY_FMT returned only listed sizes. Hans, Sakari, would you prefer this, or a different approach for the non-crop path? Thanks, Volodymyr ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output 2026-11-17 17:56 ` Volodymyr Chapman @ 2026-11-18 9:51 ` Andy Shevchenko 0 siblings, 0 replies; 6+ messages in thread From: Andy Shevchenko @ 2026-11-18 9:51 UTC (permalink / raw) To: Volodymyr Chapman Cc: hansg, mchehab, sakari.ailus, andy, gregkh, linux-media, linux-staging, linux-kernel On Tue, Nov 17, 2026 at 05:56:35PM +0000, Volodymyr Chapman wrote: > On Tue, Nov 17, 2026 at 12:31:31PM +0200, Andy Shevchenko wrote: > > So, what makes us stop implementing the crop support there? > > atomisp only uses its crop path for sensors whose driver supports the > NATIVE_SIZE, CROP_DEFAULT and CROP selection targets > (atomisp_init_sensor()). The IMX175 driver does not yet: in his review of > it, Dave Stevenson suggested that binning, and possibly cropping, should > wait for the Common Raw Sensor Model. Once the sensor supports cropping, > atomisp takes the atomisp_enum_framesizes_crop() path and this code > is not used for it. Personally to me the above sounds like "we don't accept more raw sensors until CRSM will be implemented". Because I don't see the point in implementing this hack just because something is pending. Perhaps we need to push people to tell what the estimations are for that to appear? > Even with crop support, preview mode would list nothing larger than > 1920x1440: that is the largest entry in the frame_sizes table of > atomisp_enum_framesizes_crop_inner(). This patch applies the same limit > to the non-crop path. This confuses me. Would this code be still in use after CRSM be implemented? > > No need to repeat these two paragraphs, make sure they are in the cover letter. > > Of course. I will keep them in the cover letter only in v2. > > > TBH, this feels like a hack, but I'm not v4l2/media reviewer/maintainer, > > so I don't know if this approach is legit. > > The intent is to make VIDIOC_ENUM_FRAMESIZES match what > atomisp_try_fmt() returns, which already clamps the size to 1920x1440 in > preview mode. Without this patch the IMX175 lists 3264x2448 and > 1624x1216, and TRY_FMT for 3264x2448 returns 1624x1216. With it, the > IMX175 lists only 1624x1216 and the OV2722 lists 1916x1076, 1616x1076 > and 1436x1076. Every listed size is returned unchanged by TRY_FMT, and > for the other sizes I tried (640x480 up to 3264x2448) TRY_FMT > returned only listed sizes. > > Hans, Sakari, would you prefer this, or a different approach for the > non-crop path? -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/2] media: atomisp: Report frame interval for sensors without enum_frame_interval 2026-11-16 17:57 [PATCH 0/2] media: atomisp: Fix PipeWire/Firefox streaming for sensors without crop or frame interval support Volodymyr Chapman 2026-11-16 17:57 ` [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output Volodymyr Chapman @ 2026-11-16 17:57 ` Volodymyr Chapman 1 sibling, 0 replies; 6+ messages in thread From: Volodymyr Chapman @ 2026-11-16 17:57 UTC (permalink / raw) To: Hans de Goede, Mauro Carvalho Chehab Cc: Sakari Ailus, Andy Shevchenko, Greg Kroah-Hartman, linux-media, linux-staging, linux-kernel, Volodymyr Chapman atomisp_enum_frameintervals() passes VIDIOC_ENUM_FRAMEINTERVALS on to the sensor's enum_frame_interval pad op. Most sensor drivers used with atomisp do not implement it, e.g. the OV2722 driver only implements get_frame_interval and raw sensor drivers such as the IMX175 driver set their frame rate with the pixel rate and blanking controls instead. For these sensors VIDIOC_ENUM_FRAMEINTERVALS fails, so userspace does not know the frame rate. PipeWire's V4L2 source then advertises a framerate range of 1/1 to 120/1. Firefox (libwebrtc) takes the minimum of such a range as the camera's maximum framerate and drops all but 1 frame per second, while the sensor runs at 30 fps. When the sensor has no enum_frame_interval, report one discrete frame interval: the sensor's get_frame_interval if it has that, or else the one following from its V4L2_CID_PIXEL_RATE, V4L2_CID_HBLANK and V4L2_CID_VBLANK controls and active format, rounded to a whole number of frames per second. This is the interval of the sensor's current mode, which is reported for every frame size; for the OV2722 and IMX175 all modes run at 30 fps. Tested on a Lenovo ThinkPad 10 (20C3, Atom Z3795) with this file built out of tree against a 7.2.15 kernel: ENUM_FRAMEINTERVALS reports 30 fps for both the OV2722 front sensor (from get_frame_interval) and the IMX175 rear sensor (256 MHz / (3430 * 2500)), and webcamtests.com in Firefox 160 shows about 30 fps for both instead of 1 fps. This was build-tested on top of media.git next. The problem was investigated and this patch drafted with the help of an AI coding assistant (Claude); the testing was done on the hardware by the submitter. Signed-off-by: Volodymyr Chapman <volodymyrchapman@gmail.com> --- .../staging/media/atomisp/pci/atomisp_ioctl.c | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c index 046a95857..c83834405 100644 --- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c +++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c @@ -576,6 +576,65 @@ static int atomisp_enum_framesizes(struct file *file, void *priv, return 0; } +/* + * For sensors without enum_frame_interval, get the frame interval of the + * sensor's current mode: from get_frame_interval, or else from the pixel rate + * and blanking controls which raw sensor drivers use to set the frame rate. + */ +static int atomisp_sensor_frame_interval(struct atomisp_input_subdev *input, + struct v4l2_fract *interval) +{ + struct v4l2_subdev_frame_interval fi = { + .which = V4L2_SUBDEV_FORMAT_ACTIVE, + }; + struct v4l2_subdev_format fmt = { + .which = V4L2_SUBDEV_FORMAT_ACTIVE, + }; + struct v4l2_ctrl *pixel_rate, *hblank, *vblank; + struct v4l2_subdev_state *act_sd_state; + u64 frame_size, fps; + int ret; + + act_sd_state = v4l2_subdev_lock_and_get_active_state(input->sensor); + ret = v4l2_subdev_call(input->sensor, pad, get_frame_interval, + act_sd_state, &fi); + if (ret != -ENOIOCTLCMD) { + if (act_sd_state) + v4l2_subdev_unlock_state(act_sd_state); + if (ret) + return ret; + if (!fi.interval.numerator || !fi.interval.denominator) + return -EINVAL; + *interval = fi.interval; + return 0; + } + + ret = v4l2_subdev_call(input->sensor, pad, get_fmt, act_sd_state, &fmt); + if (act_sd_state) + v4l2_subdev_unlock_state(act_sd_state); + if (ret) + return ret; + + pixel_rate = v4l2_ctrl_find(input->sensor->ctrl_handler, + V4L2_CID_PIXEL_RATE); + hblank = v4l2_ctrl_find(input->sensor->ctrl_handler, V4L2_CID_HBLANK); + vblank = v4l2_ctrl_find(input->sensor->ctrl_handler, V4L2_CID_VBLANK); + if (!pixel_rate || !hblank || !vblank) + return -EINVAL; + + frame_size = (u64)(fmt.format.width + v4l2_ctrl_g_ctrl(hblank)) * + (fmt.format.height + v4l2_ctrl_g_ctrl(vblank)); + fps = div64_u64(v4l2_ctrl_g_ctrl_int64(pixel_rate) + frame_size / 2, + frame_size); + if (!fps) + return -EINVAL; + + interval->numerator = 1; + interval->denominator = fps; + + return 0; +} + static int atomisp_enum_frameintervals(struct file *file, void *priv, struct v4l2_frmivalenum *fival) { @@ -601,6 +660,11 @@ static int atomisp_enum_frameintervals(struct file *file, void *priv, act_sd_state, &fie); if (act_sd_state) v4l2_subdev_unlock_state(act_sd_state); + if (ret == -ENOIOCTLCMD) { + if (fival->index) + return -EINVAL; + ret = atomisp_sensor_frame_interval(input, &fie.interval); + } if (ret) return ret; -- 2.43.0 ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-18 9:51 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-11-16 17:57 [PATCH 0/2] media: atomisp: Fix PipeWire/Firefox streaming for sensors without crop or frame interval support Volodymyr Chapman 2026-11-16 17:57 ` [PATCH 1/2] media: atomisp: Don't enumerate frame sizes the preview pipe can't output Volodymyr Chapman 2026-11-17 10:31 ` Andy Shevchenko 2026-11-17 17:56 ` Volodymyr Chapman 2026-11-18 9:51 ` Andy Shevchenko 2026-11-16 17:57 ` [PATCH 2/2] media: atomisp: Report frame interval for sensors without enum_frame_interval Volodymyr Chapman
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®