rpicam-apps
rpicam-apps copied to clipboard
raspberrypi
[BUG] Arducam ov7251 showing as not available
Describe the bug libcamera-hello reports that no cameras are available
Bug report See attached
Additional context I'm trying to use the Arducam ov7251 with a Raspberry Pi Zero W 2. The camera works great at 133fps in Buster on a Raspberry Pi 4 with the library and utilities found: https://github.com/ArduCAM/MIPI_Camera. I reported the issue on the Arducam forum and was directed to checkout the Bullseye upgraded libcamera support which looks to have ov7251 support.
I'm an experienced embedded systems C programmer but in over my head in the Linux kernel space and MIPI camera protocol, but would he happy to run, compile, probe, or assist in any way possible.
Thank you very much for looking at this.
ov7251 does not have a tuning file or camera_helper file in libcamera, so it won't be detected. The libcamera project has a requirement for all supported platforms to be running (or theoretically runnable) with mainline Linux kernels. The ov7251 driver isn't in mainline. Technically neither are imx477 or ov9281, but there is either a commitment or contract in place for that to happen.
ov7251 does not have a tuning file or camera_helper file in libcamera, so it won't be detected. The libcamera project has a requirement for all supported platforms to be running (or theoretically runnable) with mainline Linux kernels. The ov7251 driver isn't in mainline. Technically neither are imx477 or ov9281, but there is either a commitment or contract in place for that to happen.
Ah, ok, so it probably won't happen anytime soon. I saw the ov7251.c files the Linux kernel space and was hopeful. Thank you for the quick response. Should I close out the issue?
My apologies, ov7251 is in mainline - https://github.com/torvalds/linux/blob/master/drivers/media/i2c/ov7251.c
There are 2 small patches on top of that that aren't necessary for the sensor to work, but should be acceptable to upstream without significant issue.
e1ced9cc34b7 media: i2c: ov7251: Make the enable GPIO optional.
e3524023692d media: i2c: ov7251: Add fwnode properties controls
(I think libcamera will object if the fwnode properties aren't present)
I do have an ov7251 on my desk here, so could ask @davidplowman to do a tuning (it's only a mono sensor so not too involved). I suspect that the camera helper will be the same as ov9281.
That is great news! For learning purposes, how did you find the additional patches? How would I apply these? I'd be happy to help if pointed in the right direction.
I wrote the additional patches :-) https://github.com/raspberrypi/linux/commits/rpi-5.10.y/drivers/media/i2c/ov7251.c They're already built into any Pi kernel since December 2021 (apt should be more recent than that now).
The commit hashes don't match as I looked on an rpi-5.15.y branch, and that is currently being rebased against mainline instead of having merge commits.
For libcamera support you will need to enable the Media Controller API for the sensor. The easiest way at present would be to use dtoverlay=ov7251,media-controller=1 in config.txt.
When support is merged into libcamera we'll change the default.
Looking at the bug report, there may be more work needed in the driver to be usable with libcamera:
[0:20:38.073481713] [877] ERROR CameraSensor camera_sensor.cpp:272 'ov7251 10-0060': Mandatory V4L2 control 0x009e0902 not available
[0:20:38.073527494] [877] ERROR CameraSensor camera_sensor.cpp:272 'ov7251 10-0060': Mandatory V4L2 control 0x009e0901 not available
This corresponds to HBLANK and VBLANK. The former is not important (for control, but still needed otherwise libcamera core will fail to run), but the latter will be.
There also might be a configuration issue with the kernel driver? It fails to probe the device:
[ 933.012074] ov7251 10-0060: ov7251_write_reg: write reg error -121: reg=103, val=1
[ 933.012091] ov7251 10-0060: could not set init registers
[ 1237.565248] ov7251 10-0060: ov7251_write_reg: write reg error -121: reg=103, val=1
[ 1237.565266] ov7251 10-0060: could not set init registers
I don't have a datasheet for OV7251, but will see if I can sort out data for VBLANK and HBLANK.
I suspect it'll be the same as OV9281 in 0x380e/f, which indeed are even commented in the init sequence as /* total vertical timing [high|low] */ and 0x380c/d as /* total horiz timing high */
It also doesn't have selection API implemented for active array size. Those numbers can be made up as long as the active area is correct, but otherwise it needs the datasheet.
It streams fine for me without MC using v4l2-ctl. Pi4. No kernel errors logged. There was a rework to the overlays as part of using GPIOs or not, and the fix for the 02W fixups. The firmware and kernel versions look all up to date though, so there shouldn't be a difference there.
It streams fine for me without MC using v4l2-ctl. Pi4. No kernel errors logged. There was a rework to the overlays as part of using GPIOs or not, and the fix for the 02W fixups. The firmware and kernel versions look all up to date though, so there shouldn't be a difference there.
OMG you are right! I was able to use, "v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=Y10 --stream-mmap --stream-to=output.raw --stream-count=1" to capture a frame after setting the FPS to 90. Need to talk to the optical engineer if 90FPS will work vs. the 133 on the Pi 4.
I have the ov7251 manual and could maybe add a new reg_value for a higher FPS. Is there any reading you would recommend to help? I think the process would be to fork the ov7251.c file, add the correct register values for the higher FPS, compile the new ov7251.c as a kernel module, then do a pull request?
I'll close out the bug report since there is a working solution in v4l2. Thank you for looking into this so promptly and in depth.
Using VBLANK and HBLANK controls is the correct way to control the frame rate. They set VTS and HTS registers in the sensor respectively.
If you want to make the relevant changes then it would be very welcome. If you look at https://github.com/raspberrypi/linux/commits/rpi-5.10.y/drivers/media/i2c/imx290.c there are similar commits in the history there. Otherwise it's probably quicker for me to do it than try and explain.
I'm trying to think of a way to say, "I would like to do it but my lack of Linux kernel experience is so lacking I feel it would take days / weeks to come up to speed," without sounding lame but can't. I use Linux daily and do bare metal C daily but have never done a kernel patch or rolled one into the master branch.
:-) I'll have a look then.
@naushir Could you reopen this as we may as well add official support for it.
Hmm, ov7251 implements the set frame interval ioctl, which is not appropriate for raw sensors - https://www.kernel.org/doc/html/latest/driver-api/media/camera-sensor.html#frame-interval-configuration Mainline may go either way on retaining support for that, or implementing VBLANK/HBLANK appropriately. It should be possible to have the frame interval code set VBLANK appropriately, but is just more stuff to handle.
You made a good call on not trying to implement it!
You made a good call on not trying to implement it!
Ha, yeah, I got the feeling it would be over my head. If you have any good books or reading on the topic of kernel modules and how to manage them it would be great opportunity for me to fill in a large knowledge gap.
Kernel driver changes done for OV7251 - https://github.com/raspberrypi/linux/pull/4897 I copied the OV9281 cam_helper and tuning json file, and ov7251 has just run for me via libcamera.
Exposure time does appear to be calibrated correctly. I'm assuming gain follows the same formula as OV9281, and that appears to work fine.
That is awesome! Thank you! I can't wait to review the code and see how things were addressed and learn more about the correct way to tackle a kernel level change like this. What frame rates can are we able to run the ov7251 at?
I've allowed VBLANK to be configured up to the 90.43fps that the old driver supported. It may be possible to go faster than that, but I haven't validated the minimum vertical blanking over and above the height of the image. The Omnivision product brief says VGA@120fps, QVGA@180fps, or QQVGA@360fps. I'm not planning on adding the extra modes, but someone else may do so.
An empirical test says that VBLANK can go down to 41 with the sensor still behaving, giving a frame rate of 99.31fps. Reducing to 40 drops the frame rate to 55.48fps. I'll update the PR accordingly.
There are other parameters that can be changed in the configuration that can increase the frame rate. The HTS registers (0x380c/d) are the obvious ones that will amend the horizontal timing. The current config sets them to 0x3a0 / 928, which seems quite high considering the active frame is only 640 pixels wide. Reducing it generally seems to give a corrupted frame, so there may be other dependencies there. 0x340 / 832 seems to work to give a max rate of 110.75fps, but we're starting to tread on dodgy ground here, and need to amend the handling for frame_intervals again.
That testing has just shown up that setting the exposure range based on VTS is incorrect for VBLANK values below 45 - the max value doesn't get adjusted.
@kbingham Seeing as you've reacted to the above post, I don't suppose you have a datasheet for this sensor? It'd be nice to have the crop areas correct based on a datasheet, and know what registers 0x3016, 0x3017, 0x3018, 0x301a, 0x301b, and 0x301c do.
An empirical test says that VBLANK can go down to 41 with the sensor still behaving, giving a frame rate of 99.31fps. Reducing to 40 drops the frame rate to 55.48fps. I'll update the PR accordingly.
Too slow - Phil's merged the changes. 90fps will do for now.
@kbingham Seeing as you've reacted to the above post, I don't suppose you have a datasheet for this sensor? It'd be nice to have the crop areas correct based on a datasheet, and know what registers 0x3016, 0x3017, 0x3018, 0x301a, 0x301b, and 0x301c do.
0x3016-0x301c is SC_CLKRST0 to SC_CLKRST6
Awesome. Many thanks Kieran. The normal level of detail from Omnivision I see :-/
Looking at the defaults, HTS is set to 0x0304 / 772. Another test would be to run the sensor on defaults and see what we get. Not a priority for me at present though.
I'll close this down now, as https://github.com/raspberrypi/linux/pull/4897 has been merged.
We do need a camera_helper and basic tuning for OV7251 still - I've passed the sensor over to @davidplowman to deal with that side (just copying the ov9281 files works).
Minor note. The OV7251 gain register varies based on revision of the sensor. Using the OV9281 algorithm (gain = val/16) is only valid for revisions 1C to 1F. The driver assumes that in setting 0x3509 [4] to 1, but doesn't fail to probe on earlier revisions (it does read the rev code).
Datasheet is on our internal server, or at https://datasheetspdf.com/pdf-file/1252891/OmniVision/OV7750/1 (OV7750 is the colour version of OV7251). See reg 0x350B for gain definition (page 82).
OMG you are right! I was able to use, "v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=Y10 --stream-mmap --stream-to=output.raw --stream-count=1" to capture a frame after setting the FPS to 90.
In my case, I had to add a space in the format name like so v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat='Y10 ' --stream-mmap --stream-to=output.raw --stream-count=1. I am wondering about how to view the raw file though.
In my case, I had to add a space in the format name like so v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat='Y10 ' --stream-mmap --stream-to=output.raw --stream-count=1. I am wondering about how to view the raw file though.
Yes V4L2 have inserted spaces in some of the 4CCs, just to be awkward.
Y10 is 10bit samples packed into the low bits of 16bit words. I tend to use Vooya under Linux to view these sorts of images (or most RGB or YUV images for that matter).
Y10 is 10bit samples packed into the low bits of 16bit words. I tend to use Vooya under Linux to view these sorts of images (or most RGB or YUV images for that matter).
I went with using yuview because it is available on Raspbian with sudo apt install yuview. To view raw image I set Width to 640, Height to 480, and a custom YUV format with Chroma Subsampling set to 4:0:0, Bit depth set to 10, and Endianness set to Little Endian.
Using the the OV9281 cam_helper and tuning json file, ov7251 is working for me with kernel version 5.15.34-v71. I am however getting a highly overexposed image with the following command
libcamera-still --verbose --thumb none --immediate -o test.jpg
The following command that starts with lores capture step produces a "normal" image
libcamera-still --verbose --thumb none -o test.jpg
Any thoughts on why image without lowres capture step is overexposed?