RGBD, IMU and Lidar together not working.

[manishsaini0901]

I have all three sensors' data synced in a rosbag, I tried RGBD with IMU and it works fine with RTABMap but as soon as I give 2d Lidar data in the from of point-cloud or scan, I simply don't get any visualization anymore and after looking at rqt_graph, it seems mapping itself is not working but Odom is there. I tried tweaking parameters but still no luck.

Please let me know if additional information is needed. System - Ubuntu 20.04, Noetic RTABMap - Build from source

[manishsaini0901]

Here is the rosbag info: where I am trying to use - /uav2/pointcloud with /uav1/realsense/color/image_raw , /uav1/realsense/aligned_depth_to_color/image_raw , and /uav1/realsense/imu together

path:        merged_uav_all.bag
version:     2.0
duration:    10:04s (604s)
start:       Apr 17 2023 17:10:43.52 (1681744243.52)
end:         Apr 17 2023 17:20:48.22 (1681744848.22)
size:        185.4 GB
messages:    1688630
compression: none [112531/112531 chunks]
types:       dvs_msgs/EventArray                   [5e8beee5a6c107e504c2e78903c224b8]
             dynamic_reconfigure/Config            [958f16a05573709014982821e6822580]
             dynamic_reconfigure/ConfigDescription [757ce9d44ba8ddd801bb30bc456f946f]
             geometry_msgs/TwistStamped            [98d34b0043a2093cf9d9345ab6eef12e]
             iss_msgs/ImageInfo                    [d1102cf50605434534bb9b64cd593974]
             realsense2_camera/IMUInfo             [a02adb3a99530b11ba18a16f40f9512a]
             sbg_driver/SbgEkfEuler                [4862d08d71471abacf02c798eaa2e18f]
             sbg_driver/SbgEkfQuat                 [f1257400ac859ad475d8b651b008ba22]
             sbg_driver/SbgImuData                 [59cc541d794c367e71030fa700720826]
             sbg_driver/SbgMag                     [de7614c4cbb6cbd430c4a9b79bad88ca]
             sbg_driver/SbgStatus                  [1b73c890bd111d40339f4be9a7495e96]
             sbg_driver/SbgUtcTime                 [89495f07708fa38e487b6509c4edabaa]
             sensor_msgs/CameraInfo                [c9a58c1b0b154e0e6da7578cb991d214]
             sensor_msgs/Image                     [060021388200f6f0f447d0fcd9c64743]
             sensor_msgs/Imu                       [6a62c6daae103f4ff57a132d6f95cec2]
             sensor_msgs/MagneticField             [2f3b0b43eed0c9501de0fa3ff89a45aa]
             sensor_msgs/PointCloud2               [1158d486dd51d683ce2f1be655c3c181]
             tf2_msgs/TFMessage                    [94810edda583a504dfda3829e70d7eec]
topics:      /tf_static                                                           3 msgs    : tf2_msgs/TFMessage                   
             /uav1/realsense/accel/imu_info                                       1 msg     : realsense2_camera/IMUInfo            
             /uav1/realsense/aligned_depth_to_color/camera_info               17973 msgs    : sensor_msgs/CameraInfo               
             /uav1/realsense/aligned_depth_to_color/image_raw                 17973 msgs    : sensor_msgs/Image                    
             /uav1/realsense/color/camera_info                                17984 msgs    : sensor_msgs/CameraInfo               
             /uav1/realsense/color/image_raw                                  17984 msgs    : sensor_msgs/Image                    
             /uav1/realsense/gyro/imu_info                                        1 msg     : realsense2_camera/IMUInfo            
             /uav1/realsense/imu                                             120032 msgs    : sensor_msgs/Imu                      
             /uav1/ximea_camera_down/image_info                               44980 msgs    : iss_msgs/ImageInfo                   
             /uav1/ximea_camera_down/image_raw                                44980 msgs    : sensor_msgs/Image                    
             /uav1/ximea_nodelet_down/parameter_descriptions                      1 msg     : dynamic_reconfigure/ConfigDescription
             /uav1/ximea_nodelet_down/parameter_updates                           1 msg     : dynamic_reconfigure/Config           
             /uav2/dvxplorer_ros_driver/parameter_descriptions                    1 msg     : dynamic_reconfigure/ConfigDescription
             /uav2/dvxplorer_ros_driver/parameter_updates                         1 msg     : dynamic_reconfigure/Config           
             /uav2/events                                                     17997 msgs    : dvs_msgs/EventArray                  
             /uav2/imu                                                       484804 msgs    : sensor_msgs/Imu                      
             /uav2/pointcloud                                                  1689 msgs    : sensor_msgs/PointCloud2              
             /uav3/ellipse_micro33/imu/data                                  119977 msgs    : sensor_msgs/Imu                      
             /uav3/ellipse_micro33/imu/mag                                      600 msgs    : sensor_msgs/MagneticField            
             /uav3/ellipse_micro33/imu/velocity                              359932 msgs    : geometry_msgs/TwistStamped           
             /uav3/ellipse_micro33/sbg/ekf_euler                             119977 msgs    : sbg_driver/SbgEkfEuler               
             /uav3/ellipse_micro33/sbg/ekf_quat                              119977 msgs    : sbg_driver/SbgEkfQuat                
             /uav3/ellipse_micro33/sbg/imu_data                              119978 msgs    : sbg_driver/SbgImuData                
             /uav3/ellipse_micro33/sbg/mag                                      600 msgs    : sbg_driver/SbgMag                    
             /uav3/ellipse_micro33/sbg/status                                   600 msgs    : sbg_driver/SbgStatus                 
             /uav3/ellipse_micro33/sbg/utc_time                                 600 msgs    : sbg_driver/SbgUtcTime                
             /uav3/flir_eo/camera_info                                        29991 msgs    : sensor_msgs/CameraInfo               
             /uav3/flir_eo/image_raw                                          29991 msgs    : sensor_msgs/Image                    
             /uav3/flir_eo/spinnaker_camera_nodelet/parameter_descriptions        1 msg     : dynamic_reconfigure/ConfigDescription
             /uav3/flir_eo/spinnaker_camera_nodelet/parameter_updates             1 msg     : dynamic_reconfigure/Config

[matlabbe]

Hi you need to use rgbd_sync node to sync camera topics, then set subscribe_rgbd=true with subscribe_scan_cloud=true for rtabmap node. Remap rtabmap input rgbd_image topic to output of rgbd_sync node. approx_sync would be false for rgbd_sync, and true for rtabmap node.

[manishsaini0901]

Thanks a lot for your answer @matlabbe. It worked like a charm.