OpenCV Error by loop_fusion

[awenbigprawn]

I use VINS FUSION stereo+imu to run EuRoC with Loop closure, everything is just fine. Then I run with my own dataset, the VIO vins_node works well but after 18 seconds the loop_fusion will get an error.

In terminal I run "rosrun loop_fusion loop_fusion_node /home/yuwen/catkin_ws/src/VINS-Fusion/config/Urbant/euroc_stereo_imu_config.yaml":

config_file: /home/yuwen/catkin_ws/src/VINS-Fusion/config/Urbant/euroc_stereo_imu_config.yaml vocabulary_file/home/yuwen/catkin_ws/src/VINS-Fusion/loop_fusion/../support_files/brief_k10L6.bin loop start load vocabulary BRIEF_PATTERN_FILE/home/yuwen/catkin_ws/src/VINS-Fusion/loop_fusion/../support_files/brief_pattern.yml cam calib path: /home/yuwen/catkin_ws/src/VINS-Fusion/config/Urbant/cam0_pinhole.yaml VIO input, perfrom 4 DoF (x, y, z, yaw) pose graph optimization no previous pose graph OpenCV Error: Assertion failed (src[i].dims <= 2 && src[i].rows == src[0].rows && src[i].type() == src[0].type()) in hconcat, file /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/core/src/matrix.cpp, line 2865 terminate called after throwing an instance of 'cv::Exception' what(): /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/core/src/matrix.cpp:2865: error: (-215) src[i].dims <= 2 && src[i].rows == src[0].rows && src[i].type() == src[0].type() in function hconcat

[awenbigprawn]

my config document is like following:

%YAML:1.0

#common parameters #support: 1 imu 1 cam; 1 imu 2 cam: 2 cam; imu: 1
num_of_cam: 2

imu_topic: "/zed2/zed_node/imu/data" image0_topic: "/zed2/zed_node/left/image_rect_gray" image1_topic: "/zed2/zed_node/right/image_rect_gray" output_path: "~/output/"

cam0_calib: "cam0_pinhole.yaml" cam1_calib: "cam1_pinhole.yaml" image_width: 720 image_height: 1280

Extrinsic parameter between IMU and Camera.

estimate_extrinsic: 1 # 0 Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it. # 1 Have an initial guess about extrinsic parameters. We will optimize around your initial guess.

body_T_cam0: !!opencv-matrix rows: 4 cols: 4 dt: d data: [1, 0, 0, -0.002, 0, 1, 0, 0.023, 0, 0, 1, -0.002, 0, 0, 0, 1]

body_T_cam1: !!opencv-matrix rows: 4 cols: 4 dt: d data: [1, 0, 0, -0.002, 0, 1, 0, -0.097, 0, 0, 1, -0.002, 0, 0, 0, 1]

#Multiple thread support multiple_thread: 1

#feature traker paprameters max_cnt: 150 # max feature number in feature tracking min_dist: 30 # min distance between two features freq: 10 # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image F_threshold: 1.0 # ransac threshold (pixel) show_track: 1 # publish tracking image as topic flow_back: 1 # perform forward and backward optical flow to improve feature tracking accuracy

#optimization parameters max_solver_time: 0.04 # max solver itration time (ms), to guarantee real time max_num_iterations: 8 # max solver itrations, to guarantee real time keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

#imu parameters The more accurate parameters you provide, the better performance acc_n: 0.1 # accelerometer measurement noise standard deviation. gyr_n: 0.01 # gyroscope measurement noise standard deviation.
acc_w: 0.001 # accelerometer bias random work noise standard deviation.
gyr_w: 0.0001 # gyroscope bias random work noise standard deviation.
g_norm: 9.81007 # gravity magnitude

#unsynchronization parameters estimate_td: 0 # online estimate time offset between camera and imu td: 0.0 # initial value of time offset. unit: s. readed image clock + td = real image clock (IMU clock)

#loop closure parameters load_previous_pose_graph: 0 # load and reuse previous pose graph; load from 'pose_graph_save_path' pose_graph_save_path: "./vins_result/posegraph" # save and load path save_image: 1 # save image in pose graph for visualization prupose; you can close this function by setting 0

[awenbigprawn]

What can be the problem? My dataset is 697.47 sec, the loop closure should be detected at the end (around 680 sec). I found out that EuRoC has a loop detected at the very beginning, and shows no error in loop_fusion.

[NatureLan-sudo]

I met the same problem, have you solved it? @awenbigprawn

[awenbigprawn]

It seems like the distance between the IMU modul and the stereo camera should be larger at 1e-1m level. This means better have give a not so small last column of body_T_cam0/1. @NatureLan-sudo But it is just my case.

[NatureLan-sudo]

I have a try but it doesn't work. I foud that the parameter of euroc is small to, but it doesn't meet this problem, this is the parameter of euroc: I feel puzzled like you, because EuRoC has a loop detected at the very beginning, and shows no error in loop_fusion. But problem occuered with my dataset. @awenbigprawn