Detailed control of auto exposure (application: low light motion capture)

[oysstu]

Hi,

We are experiencing issues with the auto exposure on OAK-D in low-light situations with moderate motion. The resulting images are blurred. Would it be possible to modify the auto-exposure behavior to cap the exposure time? I've tried tweaking the exposure compensation integer, but haven't gotten the results I'd like. What does this compensation integer value do? I haven't been able to find any documentation for it.

[themarpe]

(CC: @alex-luxonis)

[alex-luxonis]

Hi @oysstu, You can find a good description of the AE compensation parameter, that matches our usage (-9 .. +9 range, 0.333 EV step, resulting -3 .. +3 EV range) here: https://developer.android.com/reference/android/hardware/camera2/CaptureRequest#CONTROL_AE_EXPOSURE_COMPENSATION We'll work on getting this better documented in our docs as well.

More details on how it adjusts the AE target exposure, depending on the value set:

• positive → applied as post-gain in ISP, i.e the sensor exposure and gain (gain=iso/100) doesn't change, but digital gain is set in ISP processing;
• negative → applied on sensor as exposure and/or gain change.

However, auto-exposure is currently implemented in the camera tuning database is by prioritizing the increase of exposure time vs. increasing the gain, as in this graph: So in low-light, first the exposure is applied until the max is reached (and this causes motion blur), then gain starts to increase. This behavior is slightly affected by the anti-banding/anti-flicker config (default enabled for 50Hz mains), that sets an exposure in multiples of 1/2 of mains period. So for 50Hz, 1/2 * 20ms = 10ms, resulting in effective exposure times of: fine-grained if < 10ms, 10ms, 20ms, 30ms.

For your use-case, either manual exposure can be used (but I understand this may not be desirable), or we could try to create a custom tuning where gain is preferred over exposure time.

[oysstu]

Great, thanks for the detailed response, that explains why the auto exposure compensation did not give the expected results. Obviously, running the AE algorithm on the hardware would be best, both due to latency and offloading computations. I'm not sure what a custom tuning entails, but if it works it's definitely an option. I'm guessing this is part of some low level hardware/firmware blob, and not the open source components?

[oysstu]

Upon further testing, it seems that the mono cameras are not able to push the exposure time low enough to avoid motion blur, without becoming excessively dark, even at ISO=1600. At exposure times below 20000us, even our brightly lit offices are fairly dark. Do you have any final recommendations before closing this issue?

[Luxonis-Brandon]

We actually have an active illumination version of OAK-D called OAK-D-Pro which may be of interest here: https://shop.luxonis.com/collections/beta-store/products/oak-d-pro

Often for fast-moving objects it is necessary to have active illumination as otherwise yes motion blur will occur. Global shutter hits all pixels at the same time, so there isn't tearing/jello - but the amount of motion during the exposure time still is what will dictate the amount of motion blur.