NAN Gradient

[Darth-Zzz]

When I use

python -W ignore run.py configs/Replica/room0.yaml

the gradients of camera_tensor in optimize_cam_in_batch become nan

[zhangganlin]

Hi, Thanks for pointing out this issue! Since this project is part of a course project, and due to the limited time period, we did not test all the parts completely, we only tested on the mapper only setting if I remember correctly.

The reason for this issue could be that when optimize the camera pose in tracker, we need to project back the pixel as a ray, and find the corresponding voxel, sometimes the ray cannot find any voxel (due to bad pose initialization), thus it could happen that the optimizer cannot get proper gradient in that case.

I planned to fix this bug but forgot, sorry for that 😅. I will try to fix that later, hopefully it would not be too long.

[zhangganlin]

Hi again, I have checked the codes a bit and sadly found that the issue was much more complex than I thought, I'm afraid it would take quite long time for me to fix it since I am also quite busy at this stage. I suggest to check the point-slam repo (https://github.com/eriksandstroem/Point-SLAM) if you just need something which can have a relatively sparse data structure for storing features, i.e. if only anchoring points in the corners of the 3D voxels, theoretically it would be very similar to this project. Sorry for the inconvenience

[Darth-Zzz]

Thanks for your prompt reply! I ran the code on replica room0 with gt_pose = True. The completion is fine but the accuracy is 20+, far greater than you reported. I checked the mesh and found some wrong mesh in the center. I wonder what may have caused this problem.