[BUG] Setting manual focus does not always work.

[LucaBonfiglioli]

Describe the bug Manual focus setter does not always work.

With the exact same code and settings, sometimes the focus is set to the desired value (130) and sometimes to 0. The behavior seems to change more frequently if I disconnect and reconnect the device, or if I set/unset the autofocus. The behavior also changes when I leave the device inactive for some minutes.

I managed to reproduce this behavior with two different devices, connected to two different computers. I also tried to change the USB port.

Sometimes I also notice an abrupt change in focal length during the acquisition of the first frames, even though the autofocus is disabled and the manual focus is set to 130. After this abrupt change, the focus sets either to 0 or to the desired value.

To Reproduce I am currently using the following pipeline. At the end of it I set the autofocus to OFF and manual focus to 130.

pipeline = dai.Pipeline()

# RGB (center) Camera
center_camera = pipeline.createColorCamera()
center_camera.setPreviewSize(1280, 800)
center_camera.setBoardSocket(dai.CameraBoardSocket.RGB)
center_camera.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
center_camera.setInterleaved(True)
center_camera.setColorOrder(dai.ColorCameraProperties.ColorOrder.RGB)

# left camera
left_camera = pipeline.createMonoCamera()
left_camera.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
left_camera.setBoardSocket(dai.CameraBoardSocket.LEFT)

# right camera
right_camera = pipeline.createMonoCamera()
right_camera.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
right_camera.setBoardSocket(dai.CameraBoardSocket.RIGHT)

# center stream
center_stream = pipeline.createXLinkOut()
center_stream.setStreamName("center")
center_camera.preview.link(center_stream.input)

# left stream
left_stream = pipeline.createXLinkOut()
left_stream.setStreamName("left")
left_camera.out.link(left_stream.input)

# right stream
right_stream = pipeline.createXLinkOut()
right_stream.setStreamName("right")
right_camera.out.link(right_stream.input)

# camera control
cam_control_in = pipeline.createXLinkIn()
cam_control_in.setStreamName("cam_control")
cam_control_in.out.link(center_camera.inputControl)

# device
device = dai.Device(pipeline, usb2Mode=False)
device.startPipeline()
device.setLogLevel(dai.LogLevel.DEBUG)

# set focus
q_control = device.getInputQueue(name="cam_control")
cam_control = dai.CameraControl()
cam_control.setManualFocus(130)
cam_control.setAutoFocusMode(dai.RawCameraControl.AutoFocusMode.OFF)
q_control.send(cam_control)

Expected behavior Camera focus should be set to the same length every time I run the script.

Screenshots Running the script - this time it set the focus to 130.

Running the exact same script again - this time the focus is wrong.

[Luxonis-Brandon]

Thanks for the thorough reporting (including example results and code used) and sorry about the trouble here @LucaBonfiglioli . Will discuss with team and circle back.

[Luxonis-Brandon]

In discussions offline, another customer mentioned that they saw this in Gen1 as well. Here's what they're seeing:

^ FYI I have the same problem on gen1 - when it doesn't work I can send the command 10 times with no change. To get it unstuck I have to switch the focus value and then switch back to the value I want. Almost like command isn't sent/run due to a variable cache. There also seems to be a race condition after pipeline init. I have to wait 100 frames before sending command. But even with all this it still doesn't always work.

So this is a good clue that this isn't a regression in our Gen2, but something that has persisted and is some underlying driver focus bug. We're looking into it.

[alex-luxonis]

Yeah, we noticed that as well with autofocus initially enabled, sometimes it needs more than one set command to get to take effect. It's some bug on the FW side.

For now, could you set manual focus initially at pipeline creation time, this should fix that behavior. Like:

center_camera.initialControl.setManualFocus(130)

# This may be redundant when setManualFocus is used
center_camera.initialControl.setAutoFocusMode(dai.RawCameraControl.AutoFocusMode.OFF) 

And if you want to set other types of controls at pipeline creation time, need to use the develop branch. We'll probably make a release as well next week, to include it in main.

[LucaBonfiglioli]

Thank you, I will definitely try that next week.

Another thing that might give some clue on the reported bug: for now I got it "fixed" by setting a warmup period after device creation in which 20 frames are acquired and then discarded. After the warmup, setting the focus with the usual method seems to work consistently.

When setting a shorter warmup, like 1 to 3 frames, the bug re-appears again.

[LucaBonfiglioli]

Yeah, we noticed that as well with autofocus initially enabled, sometimes it needs more than one set command to get to take effect. It's some bug on the FW side.

For now, could you set manual focus initially at pipeline creation time, this should fix that behavior. Like:

center_camera.initialControl.setManualFocus(130)

# This may be redundant when setManualFocus is used
center_camera.initialControl.setAutoFocusMode(dai.RawCameraControl.AutoFocusMode.OFF) 

And if you want to set other types of controls at pipeline creation time, need to use the develop branch. We'll probably make a release as well next week, to include it in main.

Using initialControl instead of "camControl" input queue seems to fix the issue. Thank you.

[Grumpy141]

I have now confirmed you can switch between auto-focus and manual lens control while pipeline is running. However, when switching to manual focus, it often errors out and does not go to the correct lens position. If I pause for 2 seconds, and send again, it will then go to the correct manual lens position.

[alex-luxonis]

We have a work-in-progress branch with a workaround for this issue: https://github.com/luxonis/depthai-python/pull/336 https://github.com/luxonis/depthai-core/pull/191