Essential matrix has NaN numbers

[dannyway03]

Hi Johannes,

thanks for sharing your work! I got curious about your approach, and tried to get some pose estimation stacking libviso for feature matching and momo, following the guidelines you provided. However, I constantly crash against a runtime error saying "essential_matrix_calculator: essential matrix has nan members". I wonder if you have any idea of the reason of this, or better if you could share some working example code (like the one you used for the video).

thanks a lot! Danny

[johannes-graeter]

Hi Danny,

thanks for the interest in my work :) This happens if the extrinsic calibration you set for the cam_ is not a invertible. All zeros f.e. would result in NaNs for the calculated camera motion hence Nans in the essential matrix. Added assert for this. Does that help?

In times of Corona with some much time to spend I am writing python bindings :) At the moment for my other project https://github.com/johannes-graeter/limo , but eventually also momo. The strong side of this approach is multi sensor setups (surround setup or similar.). Unfortunately I do no longer have access to the data I used for the video (which was done during my PhD...) Do you know any good open datasets for this? I would try to make it work on those then :)

Best, Johannes

[dannyway03]

Hi Johannes,

thanks for your reply. It turned out that everything was setup correctly (as extrinsics i have identity rotation matrix and 0 translation vector), however to make it work I had to modify essential_matrix_calculator.hpp.

In the process() function, I replaced
Matrix3<T> essential_matrix = motion_cam.rotation().inverse() * commons::GetCrossProdMatrix(motion_cam.translation().eval()); with // calculate essential matrix Matrix3<T> essential_matrix = motion_cam.linear().inverse() * commons::GetCrossProdMatrix(motion_cam.translation().eval());

Does this make any sense? Can this be due to different Eigen version?

As for the dataset, i believe that a minimal working example based on Kitti would help a lot. There you can get "ground-truth" scale prior as well.

A side question, is there any way to get together with the pose, also the inlier matches?

thanks a lot! Danny

[johannes-graeter]

Hi Danny,

thx a lot for working on this! The difference between the linear part and the rotation of a Homogenous Transform is shearing and scaling parameters. Since we have an Isometry here there should be no scaling and shearing though... I could also not find a definition of rotation, all I can say is that it worked as of 1.5 years ago :)

If this worked for you I think it is safer to use this one, could you file a pull request so that the contribution is documented :)?

[johannes-graeter]

For the outlier detection I open a new issue

[dannyway03]

I currently have some problems in pushing the changes. attached you can find a patch to be applied to master. essentialNaNPatch.patch.txt

[johannes-graeter]

better late than never I just committed your changes to master :)