OV9782 camera: after being ISP scaled it's not divisible by 2 anymore

[ostepan2006]

Hello. I had been trying to calibrate my OAK-FFC-3P device. And i got an error (see print screen).

If i replace OAK-FFC-3P by OAK-D and run camera_preview.py I got another error : File "camera_preview.py", line 36, in device.startPipeline(pipeline) RuntimeError: Camera(4) - 'video' width or height (-1, -1) must be bigger than (32, 32).

I have tried cam_test.py. I got this output Luxonis-Erik posted this message: https://discord.com/channels/790680891252932659/924798475030392922/1082672913032949921

Please help me calibrate my OAK-FFC-3P device

[Erol444]

Hi @ostepan2006 , The culprit of this error is here; so these lines:

        if not self.args.disableRgb:
            rgb_cam = pipeline.createColorCamera()
            rgb_cam.setResolution(
                dai.ColorCameraProperties.SensorResolution.THE_4_K)
            rgb_cam.setInterleaved(False)
            rgb_cam.setBoardSocket(dai.CameraBoardSocket.RGB)
            rgb_cam.setIspScale(1, 3)
            rgb_cam.initialControl.setManualFocus(self.focus_value)
            rgb_cam.setFps(self.args.fps)

So it tries to use 4K resolution, fails as it's OV9782, then downscales the (default) 800P resolution by 1/3 => 427x267, which isn't supported by opencv (nv12->rgb). As a workaround, you can just remove the rgb_cam.setIspScale(1, 3) and it should work, but @njezersek we should update this so it works for users, just as we did for factory calib. Thoughts? Thanks, Erik

[njezersek]

@ostepan2006 please try switching to modular-calibration branch. This branch has the latest version of the calibration script and I think the issue is already fixed there.

[ostepan2006]

Hi @ostepan2006 , The culprit of this error is here; so these lines:

        if not self.args.disableRgb:
            rgb_cam = pipeline.createColorCamera()
            rgb_cam.setResolution(
                dai.ColorCameraProperties.SensorResolution.THE_4_K)
            rgb_cam.setInterleaved(False)
            rgb_cam.setBoardSocket(dai.CameraBoardSocket.RGB)
            rgb_cam.setIspScale(1, 3)
            rgb_cam.initialControl.setManualFocus(self.focus_value)
            rgb_cam.setFps(self.args.fps)

So it tries to use 4K resolution, fails as it's OV9782, then downscales the (default) 800P resolution by 1/3 => 427x267, which isn't supported by opencv (nv12->rgb). As a workaround, you can just remove the rgb_cam.setIspScale(1, 3) and it should work, but @njezersek we should update this so it works for users, just as we did for factory calib. Thoughts? Thanks, Erik

Thanks for your reply. After doing that i got another error.

[ostepan2006]

@ostepan2006 please try switching to modular-calibration branch. This branch has the latest version of the calibration script and I think the issue is already fixed there.

Thanks for your reply. Should I delete main branch Depthai and clone modular calibration branch? Or it might be possible use only calibrate.py script from modular calibration branch?

[Erol444]

Hi @ostepan2006 best to try the modular-calibration branch.

[ostepan2006]

Hi @ostepan2006 best to try the modular-calibration branch.

I had swithed to modular-calibration branch. After running command python calibrate.py -s 2.5 -db -brd ACME01.json -ih i got this output (see screenshot)

I had used this board configuration (see screenshot): Please , give me a tip how can i calibrate my OAK-FFC-3P device.

[njezersek]

@ostepan2006 you have to specify the cameras you have connected. I suggest you modify this file to fit your needs.

[ostepan2006]

@ostepan2006 you have to specify the cameras you have connected. I suggest you modify this file to fit your needs.

I have made my board configuration from your documentation (see screenshot below)
If my board configuration is wrong give me please a tip how board configuration have to be for OAK-FFC-3P device. I have two Global Shutter Grayscale OV9282 cameras and one Global Shutter Color OV9782 camera in my OAK-FFC-3P device. Thanks

[njezersek]

@ostepan2006 I see the link I wanted to post in the previous comment didn't display correctly. Please use this file for reference and just change the extrinsics and fovs of the cameras.

{
    "board_config":
    {
        "name": "OAK-D-PRO",
        "revision": "R3M1E3",
        "cameras":{
            "CAM_A": {
                "name": "rgb",
                "hfov": 68.7938,
                "type": "color"
            },
            "CAM_B": {
                "name": "left",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_C",
                    "specTranslation": {
                        "x": -7.5,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            },
            "CAM_C": {
                "name": "right",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_A",
                    "specTranslation": {
                        "x": 3.75,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            }
        },
        "stereo_config":{
            "left_cam": "CAM_B",
            "right_cam": "CAM_C"
        }
    }
}

[ostepan2006]

@ostepan2006 I see the link I wanted to post in the previous comment didn't display correctly. Please use this file for reference and just change the extrinsics and fovs of the cameras.

{
    "board_config":
    {
        "name": "OAK-D-PRO",
        "revision": "R3M1E3",
        "cameras":{
            "CAM_A": {
                "name": "rgb",
                "hfov": 68.7938,
                "type": "color"
            },
            "CAM_B": {
                "name": "left",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_C",
                    "specTranslation": {
                        "x": -7.5,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            },
            "CAM_C": {
                "name": "right",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_A",
                    "specTranslation": {
                        "x": 3.75,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            }
        },
        "stereo_config":{
            "left_cam": "CAM_B",
            "right_cam": "CAM_C"
        }
    }
}

Am i right in thinking that extrinsics should be filled before calibration? I thought that extrinsics will be calculated during calibration.

[njezersek]

These are so called spec extrinsics and are sometimes more accurate than the calculated ones. For example the StereoDepth node uses spec extrinsics to convert disparity to depth by default (you can change this behaviour with setDisparityToDepthUseSpecTranslation but you need to use the latest develop of DepthAI). That's why you need to provide accurate manual measurements as well.

[ostepan2006]

These are so called spec extrinsics and are sometimes more accurate than the calculated ones. For example the StereoDepth node uses spec extrinsics to convert disparity to depth by default (you can change this behaviour with setDisparityToDepthUseSpecTranslation but you need to use the latest develop of DepthAI). That's why you need to provide accurate manual measurements as well.

As I understand spec extrinsics is not mundatory to do calibration of OAK-FFC-3P device. Right? Could I live rotation fields empty (equal zero) in the board_config file? If not do you have more detailed explanation of the procedure of formation of the board_config file? In particalar it is iteresting how can I calculate all extrinsics. Thanks

[njezersek]

You can set all rotation fields to zero. I'm not sure what happens if you leave them empty. The camera without the extrinsics field set is considered to be at (0,0,0) with rotation (0,0,0). Extirnsics of other cameras must be specified relative to a camera specified in the to_cam field.

I assume your cameras are placed in the following way:

(RIGHT)------(CENTER)-------(LEFT)

and the distance between left and right camera is 5.2 cm and the distance between center and left camera is 2.6 cm.

In that case, the board config file should look something like that:

{
    "board_config":
    {
        "cameras":{
            "CAM_A": {
                "name": "rgb",
                "hfov": 68.7938,
                "type": "color"
            },
            "CAM_B": {
                "name": "left",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_C",
                    "specTranslation": {
                        "x": -5.2,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            },
            "CAM_C": {
                "name": "right",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_A",
                    "specTranslation": {
                        "x": 2.6,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            }
        },
        "stereo_config":{
            "left_cam": "CAM_B",
            "right_cam": "CAM_C"
        }
    }

[ostepan2006]

Hi @njezersek . I have created board_config.json as you had recommended. The file board_config.json is shown bellow. And I had run command python calibrate.py -s 2.5 -db -brd board_config.json -ih . ` Calibration procedure had been performing (a file calibration_output.txt is attached). But there was no calibration data on the device afterwards calibration procedure. Concretely I had had that error at the end of calibration procedure -

Device closed in exception..
'name'
Traceback (most recent call last):
  File "calibrate.py", line 925, in calibrate
    calibration_handler.setBoardInfo(self.board_config['name'], self.board_config['revision'])
KeyError: 'name'

What is a problem I had while calibration procedure? Please give me a help. Thanks

board_config.json:

{
    "board_config":
    {
        "cameras":{
            "CAM_A": {
                "name": "rgb",
                "hfov": 68.7938,
                "type": "color"
            },
            "CAM_B": {
                "name": "left",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_C",
                    "specTranslation": {
                        "x": -5.2,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            },
            "CAM_C": {
                "name": "right",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_A",
                    "specTranslation": {
                        "x": 2.6,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            }
        },
        "stereo_config":{
            "left_cam": "CAM_B",
            "right_cam": "CAM_C"
        }
    }
}

calibration_output.txt

[Erol444]

@ostepan2006 your json needs these 2 fields as well inside board_config: name and revision, just like in json above:

{
    "board_config":
    {
        "name": "OAK-D-PRO",
        "revision": "R3M1E3",
        "cameras":{
...

[ostepan2006]

Thanks. I will fix my config file and redo calibration.

[ostepan2006]

@Erol444 The previous error have not arised while calibration. But I got new error:

Reprojection error threshold -> 3
rgb Reprojection Error: 55.234071
800
Reprojection error threshold -> 1.1111111111111112
left Reprojection Error: 55.754000
800
Reprojection error threshold -> 1.1111111111111112
right Reprojection Error: 55.477004
['high Reprojection Error', 'high Reprojection Error', 'Epiploar validation failed between left and right', 'high Reprojection Error', 'Epiploar validation failed between right and rgb']

I think that there is a mismatch between cameras placement and info in configuration file. Do you have tutorial about: how do right cameras placement on the tripod relatively each other; how do right cameras placement on the tripod relatively up and down.

[Erol444]

@ostepan2006 It could be the camera placement / configuration file. What's your current setup? Could you maybe take a photo of all 3 cameras, and draw the distances between them? Thanks, Erik

[ostepan2006]

@Erol444

Thanks for the quick reply. The image of my installation is below. As you can see a distance between neighbor cameras is about 26 mm. All cameras have rotated on 180 degrees. The color camera is in between grayscale ones. Maybe I have mismatch between the grayscale cameras in the config file?

[Erol444]

@ostepan2006 based on the error it does look like the config file (json) is incorrect.