libcamera
libcamera copied to clipboard
raspberrypi
Auto Exposure for OV9281
Hello,
Using GStreamer libcamerasrc with OV9281, but ae-enable=true doesn't seem to have any effect (v0.5.1+100-e53bdf1f).
/usr/share/libcamera/ipa/rpi/pisp/ov9281_mono.json include rpi.agc, is this related?
Thank you.
Could you expand on how you are testing with AE enabled/disabled, i.e. what is the target scene like? Does rpicam-apps or picamera2 work as expected?
@naushir you're right, rpicam-still -v shows that exposure is being lowered down to 95µs automatically but output is still over exposed, is it possible that OV9281 global shutter sensor is too sensitive for a bright outdoor environment?
I think this might be a driver related issue. What does rpicam-hello --list-cameras -v output?
$ rpicam-hello --list-cameras -v
Available cameras
-----------------
0 : ov9281 [1280x800 10-bit MONO] (/base/soc/i2c0mux/i2c@1/ov9281@60)
Modes: 'R8' : 640x400 [309.79 fps - (0, 0)/1280x800 crop]
1280x720 [171.79 fps - (0, 0)/1280x720 crop]
1280x800 [143.66 fps - (0, 0)/1280x800 crop]
'R10_CSI2P' : 640x400 [247.83 fps - (0, 0)/1280x800 crop]
1280x720 [137.42 fps - (0, 0)/1280x720 crop]
1280x800 [114.93 fps - (0, 0)/1280x800 crop]
Available controls for 1280x800 R8 mode:
----------------------------------------
AeConstraintMode : [0..3]
AeEnable : [false..true]
AeExposureMode : [0..3]
AeFlickerMode : [0..1]
AeFlickerPeriod : [100..1000000]
AeMeteringMode : [0..3]
AnalogueGain : [1.000000..15.937500]
AnalogueGainMode : [0..1]
Brightness : [-1.000000..1.000000]
CnnEnableInputTensor : [false..true]
Contrast : [0.000000..32.000000]
ExposureTime : [7..8571026]
ExposureTimeMode : [0..1]
ExposureValue : [-8.000000..8.000000]
FrameDurationLimits : [6961..8575123]
HdrMode : [0..4]
NoiseReductionMode : [0..4]
ScalerCrop : [(0, 0)/64x64..(0, 0)/1280x800]
Sharpness : [0.000000..16.000000]
StatsOutputEnable : [false..true]
SyncFrames : [1..1000000]
SyncMode : [0..2]
Can you try setting manual exposure/gain values and see if the brightness changes at all?
rpicam-hello -t 0 --shutter 1ms --gain 1
rpicam-hello -t 0 --shutter 10ms --gain 1
rpicam-hello -t 0 --shutter 100ms --gain 1
With these commands, what is the reported shutter time used (on the titlebar)?
Headless (no GUI), so using rpicam-still and setting shutter manually - works, but even at the fastest - outdoor is always over exposed:
$ rpicam-still -v -o image5.jpg --timeout 500 --shutter 100us
[0:43:29.244641721] [14104] INFO Camera camera_manager.cpp:326 libcamera v0.5.1+100-e53bdf1f
[0:43:29.293472740] [14107] WARN RPiSdn sdn.cpp:40 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:43:29.294514795] [14107] INFO RPI vc4.cpp:440 Registered camera /base/soc/i2c0mux/i2c@1/ov9281@60 to Unicam device /dev/media1 and ISP device /dev/media0
[0:43:29.294605018] [14107] INFO RPI pipeline_base.cpp:1107 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
Options:
verbose: 2
info_text:#%frame (%fps fps) exp %exp ag %ag dg %dg
timeout: 500ms
width: 0
height: 0
output: image5.jpg
post_process_file:
post_process_libs:
preview: default
qt-preview: 0
transform: identity
roi: all
shutter: 100us
metering: centre
exposure: normal
ev: 0
awb: auto
flush: false
wrap: 0
brightness: 0
contrast: 1
saturation: 1
sharpness: 1
framerate: 30
denoise: auto
viewfinder-width: 0
viewfinder-height: 0
tuning-file: (libcamera)
lores-width: 0
lores-height: 0
lores-par: 0
autofocus-range: normal
autofocus-speed: normal
autofocus-window: all
hdr: off
mode: unspecified
viewfinder-mode: unspecified
metadata:
metadata-format: json
Made DRM preview window
No connector ID specified. Choosing default from list:
Connector 33 (crtc 0): type 11, 0x0
Connector 42 (crtc 0): type 11, 0x0
Preview window unavailable
Running without preview window
Opening camera...
Acquired camera /base/soc/i2c0mux/i2c@1/ov9281@60
Configuring viewfinder...
Viewfinder size chosen is 640x400
Mode selection for 640:400:12:P
R8,640x400/0 - Score: 2000
R8,1280x720/0 - Score: 2306.67
R8,1280x800/0 - Score: 2260
R10_CSI2P,640x400/0 - Score: 1000
R10_CSI2P,1280x720/0 - Score: 1306.67
R10_CSI2P,1280x800/0 - Score: 1260
Stream configuration adjusted
[0:43:29.298957277] [14104] INFO Camera camera.cpp:1205 configuring streams: (0) 640x400-YUV420/sYCC (1) 640x400-R10_CSI2P/RAW
[0:43:29.299330740] [14107] INFO RPI vc4.cpp:615 Sensor: /base/soc/i2c0mux/i2c@1/ov9281@60 - Selected sensor format: 640x400-Y10_1X10 - Selected unicam format: 640x400-Y10P
Camera streams configured
Available controls:
AeExposureMode : [0..3]
Contrast : [0.000000..32.000000]
AeConstraintMode : [0..3]
ExposureTimeMode : [0..1]
HdrMode : [0..4]
AeMeteringMode : [0..3]
AeFlickerPeriod : [100..1000000]
AnalogueGainMode : [0..1]
SyncFrames : [1..1000000]
SyncMode : [0..2]
AnalogueGain : [1.000000..15.937500]
StatsOutputEnable : [false..true]
FrameDurationLimits : [4035..10636987]
Brightness : [-1.000000..1.000000]
ScalerCrop : [(0, 0)/128x128..(0, 0)/1280x800]
ExposureTime : [9..10631860]
NoiseReductionMode : [0..4]
Sharpness : [0.000000..16.000000]
AeEnable : [false..true]
ExposureValue : [-8.000000..8.000000]
AeFlickerMode : [0..1]
CnnEnableInputTensor : [false..true]
Buffers allocated and mapped
Viewfinder setup complete
Requests created
Using crop (main) (0, 0)/1280x800
[0:43:29.303181388] [14110] WARN IPARPI ipa_base.cpp:1196 Could not set SHARPNESS - no sharpen algorithm
Camera started!
Viewfinder frame 0
#7 (0.00 fps) exp 95.00 ag 8.00 dg 1.05
Viewfinder frame 1
#8 (30.00 fps) exp 95.00 ag 8.00 dg 1.05
Viewfinder frame 2
#9 (30.00 fps) exp 95.00 ag 8.00 dg 1.05
Viewfinder frame 3
#10 (30.00 fps) exp 95.00 ag 8.00 dg 1.05
Viewfinder frame 4
#11 (30.00 fps) exp 95.00 ag 8.00 dg 1.05
Viewfinder frame 5
#12 (30.00 fps) exp 95.00 ag 8.00 dg 1.05
Viewfinder frame 6
#13 (30.00 fps) exp 95.00 ag 8.00 dg 1.05
Viewfinder frame 7
Camera stopped!
Tearing down requests, buffers and configuration
Configuring still capture...
Mode selection for 1280:800:12:P
R8,640x400/0 - Score: 4080
R8,1280x720/0 - Score: 2226.67
R8,1280x800/0 - Score: 2000
R10_CSI2P,640x400/0 - Score: 3080
R10_CSI2P,1280x720/0 - Score: 1226.67
R10_CSI2P,1280x800/0 - Score: 1000
Stream configuration adjusted
[0:43:29.849943869] [14104] INFO Camera camera.cpp:1205 configuring streams: (0) 1280x800-YUV420/sYCC (1) 1280x800-R10_CSI2P/RAW
[0:43:29.850848221] [14107] INFO RPI vc4.cpp:615 Sensor: /base/soc/i2c0mux/i2c@1/ov9281@60 - Selected sensor format: 1280x800-Y10_1X10 - Selected unicam format: 1280x800-Y10P
Camera streams configured
Available controls:
AeExposureMode : [0..3]
Contrast : [0.000000..32.000000]
AeConstraintMode : [0..3]
ExposureTimeMode : [0..1]
HdrMode : [0..4]
AeMeteringMode : [0..3]
AeFlickerPeriod : [100..1000000]
AnalogueGainMode : [0..1]
SyncFrames : [1..1000000]
SyncMode : [0..2]
AnalogueGain : [1.000000..15.937500]
StatsOutputEnable : [false..true]
FrameDurationLimits : [8701..10718904]
Brightness : [-1.000000..1.000000]
ScalerCrop : [(0, 0)/64x64..(0, 0)/1280x800]
ExposureTime : [9..10713782]
NoiseReductionMode : [0..4]
Sharpness : [0.000000..16.000000]
AeEnable : [false..true]
ExposureValue : [-8.000000..8.000000]
AeFlickerMode : [0..1]
CnnEnableInputTensor : [false..true]
Buffers allocated and mapped
Still capture setup complete
Requests created
Using crop (main) (0, 0)/1280x800
[0:43:29.854392147] [14113] WARN IPARPI ipa_base.cpp:1196 Could not set SHARPNESS - no sharpen algorithm
Camera started!
Camera stopped!
Still capture image received
Exposure time: 95
Ag 8 Dg 1.04575 Total 8.36601
Thumbnail dimensions are 320 x 240
Thumbnail size 11134
JPEG size is 398396
EXIF data len 298
Saved image 1280 x 800 to file image5.jpg
Closing RPiCam application(frames displayed 7, dropped 0)
Camera stopped!
Tearing down requests, buffers and configuration
Camera closed
Found someone reporting a similar issue back in 2020...
Could you run all three commands I've listed above and report the gain and exposure used please?
Of course:
$ rpicam-hello -t 0 --shutter 1ms --gain 1
[0:09:21.363620053] [1151] INFO Camera camera_manager.cpp:326 libcamera v0.5.1+100-e53bdf1f
[0:09:21.387072719] [1156] WARN RPiSdn sdn.cpp:40 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:09:21.388160349] [1156] INFO RPI vc4.cpp:440 Registered camera /base/soc/i2c0mux/i2c@1/ov9281@60 to Unicam device /dev/media1 and ISP device /dev/media2
[0:09:21.388237645] [1156] INFO RPI pipeline_base.cpp:1107 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
Made DRM preview window
Preview window unavailable
Mode selection for 640:400:12:P
R8,640x400/0 - Score: 2000
R8,1280x720/0 - Score: 2306.67
R8,1280x800/0 - Score: 2260
R10_CSI2P,640x400/0 - Score: 1000
R10_CSI2P,1280x720/0 - Score: 1306.67
R10_CSI2P,1280x800/0 - Score: 1260
Stream configuration adjusted
[0:09:21.391031349] [1151] INFO Camera camera.cpp:1205 configuring streams: (0) 640x400-YUV420/sYCC (1) 640x400-R10_CSI2P/RAW
[0:09:21.391394015] [1156] INFO RPI vc4.cpp:615 Sensor: /base/soc/i2c0mux/i2c@1/ov9281@60 - Selected sensor format: 640x400-Y10_1X10 - Selected unicam format: 640x400-Y10P
[0:09:21.394530460] [1159] WARN IPARPI ipa_base.cpp:1196 Could not set SHARPNESS - no sharpen algorithm
#0 (0.00 fps) exp 994.00 ag 1.00 dg 1.01
#1 (30.00 fps) exp 994.00 ag 1.00 dg 1.01
#2 (30.00 fps) exp 994.00 ag 1.00 dg 1.01
...
$ rpicam-hello -t 0 --shutter 10ms --gain 1
...
#0 (0.00 fps) exp 9992.00 ag 1.00 dg 1.00
#1 (30.00 fps) exp 9992.00 ag 1.00 dg 1.00
#2 (30.00 fps) exp 9992.00 ag 1.00 dg 1.00
...
$ rpicam-hello -t 0 --shutter 100ms --gain 1
...
#0 (0.00 fps) exp 33086.00 ag 1.00 dg 3.02
#1 (30.00 fps) exp 33086.00 ag 1.00 dg 1.00
#2 (30.00 fps) exp 33086.00 ag 1.00 dg 1.00
...
Ok, that shows the exposures set can be used. Are you able to run rpicam-still with the exact same parameters and see what the output images look like for all 3 sets?
Yes, exp 33086.00 ag 1.00 dg 1.00 is the most over exposed, then exp 9992.00 ag 1.00 dg 1.00, then exp 994.00 ag 1.00 dg 1.00, and the least over exposed is exp 95.00 ag 1.00 dg 1.00 (the image I've attached earlier).
Can you take a picture of the scene with, say, a mobile phone camera? Or alternatively can you use one of the Raspberry Pi cameras if you have one available?
This looks to be related to your OV9281 module, either hardware or software I cannot tell. Can you try another camera (preferably an official Raspberry Pi module) on your Pi and repeat the experiment.
Still waiting for a response from Waveshare. Couldn't understand if their OV9281-110 has/should have an IR filter or not. After inspecting the unit I have (unscrewed the lens) - couldn't see any filter installed, should an IR filter eliminate this "outdoors overexposed even at minimum shutter" issue? Where can I get such filter?
Thanks!
I've got an Innomaker OV9281 module here. It is a sensitive sensor as it has larger pixels than most (3um vs 1.55um for HQ camera, although 3.45um for the Pi Global shutter camera)..
Minimum exposure time is 9.56us. Capturing from my desk with fixed gain x1 gives me these images. At 50us you can just about make out the details in the clouds out the window. V4L2 exposure control value checked, and 9us gives a value of 1, with 50us being 5, so we are at the low end.
I've checked the datasheet and it does say minimum exposure time is 1 row period, and we are setting that, so there aren't really any options to reduce the exposure time further. Switching to 8 bit readout does reduce the minimum exposure to 7.65usec as it has a pixel rate of 200MPix/s instead of 160MPix/s for 10bit, and a fixed line length of 1530 pixels.
You could use a neutral density filter to reduce the sensitivity without compromising the image. https://en.wikipedia.org/wiki/Neutral-density_filter
The sensor sensitivity does not seem to be the problem here based on the image captured with your iphone 12 capture. Are you running a stock raspberry pi kernel (and OV9281 driver)?
@naushir Yes. Outdoors (sunlight) overexpuse was due to the lack of IR filter, which wasn't mentioned in the product page/documentation. AFAIK unmentioning this indicates the existance of an IR filter, manufacturers/resellers must state "NoIR" otherwise.
BTW, why won't libcamerasrc output GRAY8 format for OV9281?
BTW, why won't libcamerasrc output GRAY8 format for OV9281?
I'm not sure if gstreamer natively knows about the greyscale formats. Resolving this now...
BTW, why won't libcamerasrc output GRAY8 format for OV9281?
I'm not sure if gstreamer natively knows about the greyscale formats. Resolving this now...
GStreamer's core knows about 8bit mono images (GST_VIDEO_FORMAT_GRAY8), but libcamera (and hence libcamerasrc) is only set up for producing colour images. The simplest approach is to ask for YUV420 and ignore the chroma planes (which should all be U=128 V=128 anyway as the image is mono).
The simplest approach is to ask for YUV420 and ignore the chroma planes (which should all be U=128 V=128 anyway as the image is mono).
Seeing as we already have the format tweaking on pisp for RGB outputs, can we do similar for R8 mono? Just don't pass out the chroma plane handles. Doing it on vc4 would require overallocated buffers as it only supports one buffer handle for all the planes.
Hello,
Related issue, it seems that libcamerasrc is outputting GRAY16_LE format but why with "RGB" colorimetry?
$ GST_DEBUG=4 gst-launch-1.0 libcamerasrc ! video/x-raw,format=GRAY16_LE,width=640,height=400,framerate=30/1 ! fakesink
...
[17056] INFO Camera camera.cpp:1215 configuring streams: (0) 640x400-R16/RAW
[17052] INFO RPI pisp.cpp:1483 Sensor: /base/axi/pcie@1000120000/rp1/i2c@88000/ov9281@60 - Selected sensor format: 640x400-Y10_1X10/RAW - Selected CFE format: 640x400-Y16 /RAW/Linear/Rec601/Limited
17048 0x7fff00000b70 INFO GST_EVENT gstevent.c:912:gst_event_new_caps: creating caps event video/x-raw, format=(string)GRAY16_LE, width=(int)640, height=(int)400, colorimetry=(string)1:1:1:0, framerate=(fraction)30/1
17048 0x7fff00000b70 INFO basetransform gstbasetransform.c:1326:gst_base_transform_setcaps:<capsfilter0> reuse caps
17048 0x7fff00000b70 INFO GST_EVENT gstevent.c:912:gst_event_new_caps: creating caps event video/x-raw, format=(string)GRAY16_LE, width=(int)640, height=(int)400, colorimetry=(string)1:1:1:0, framerate=(fraction)30/1
17048 0x7fff00000b70 WARN video-info video-info.c:190:validate_colorimetry: color matrix RGB is only supported with RGB format, GRAY16_LE is not
Thanks.