introlab
Issue with running RTAB-Map on AGX Orin
Hi...
Installation and Setup: We installed the rtabmap_ros package on AGX Orin using the source link below and followed the instructions to build it with multiple RGB-D cameras: source link: GitHub - introlab/rtabmap_ros at humble-devel
Issue Description: When attempting to launch the file "realsense_d400.launch.py", we encountered the following errors related to synchronization issues and out-of-memory errors.
Error Logs:
[WARN] [1741009439.747586448] [rtabmap_viz]: rtabmap_viz: Did not receive data since 5 seconds! Make sure the input topics are published ("$ ros2 topic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). Ajusting topic_queue_size (20) and sync_queue_size (20) can also help for better synchronization if framerates and/or delays are different. Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. [rtabmap_viz-4] rtabmap_viz subscribed to (exact sync): [rtabmap_viz-4] /odom \ [rtabmap_viz-4] /camera/camera/color/image_raw \ [rtabmap_viz-4] /camera/camera/aligned_depth_to_color/image_raw \ [rtabmap_viz-4] /camera/camera/color/camera_info \ [rtabmap_viz-4] /odom_info
terminate called after throwing an instance of 'cv::Exception' [rgbd_odometry-2] what(): OpenCV(4.10.0-dev) /home/orin-5/opencv/modules/core/src/alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 55505507871600 bytes in function 'OutOfMemoryError' [rgbd_odometry-2] [ERROR] [rgbd_odometry-2]: process has died [pid 75066, exit code -6, cmd '/home/orin-5/RTABMAP_ws/install/rtabmap_odom/lib/rtabmap_odom/rgbd_odometry --ros-args -p depth_module.emitter_enabled:=1 --params-file /tmp/launch_params_4_hoy6sq -r rgb/image:=/rs/rgb_image_sync -r rgb/camera_info:=/rs/rgb_camera_info_sync -r depth/image:=/rs/aligned_depth_image_sync']. [rtabmap_viz-4] [WARN] [1741163153.488268078] [rtabmap_viz]: rtabmap_viz: Did not receive data since 5 seconds! Make sure the input topics are published ("$ ros2 topic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). Ajusting topic_queue_size (10) and sync_queue_size (10) can also help for better synchronization if framerates and/or delays are different. If topics are not published at the same rate [rgbd_odometry-2] [WARN] [1741163152.484306351] [rgbd_odometry]: The time difference between rgb and depth frames is high (diff=0.033919s, rgb=1741163152.302351s, depth=1741163152.268432s). You may want to set approx_sync_max_interval lower than 0.02s to reject spurious bad synchronizations or use approx_sync=false if streams have all the exact same timestamp. [rgbd_odometry-2] terminate called after throwing an instance of 'cv::Exception' [rgbd_odometry-2] what(): OpenCV(4.10.0-dev) /home/orin-5/opencv/modules/core/src/alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 55505507871600 bytes in function 'OutOfMemoryError' [rgbd_odometry-2] [ERROR] [rgbd_odometry-2]: process has died [pid 75066, exit code -6, cmd '/home/orin-5/RTABMAP_ws/install/rtabmap_odom/lib/rtabmap_odom/rgbd_odometry --ros-args -p depth_module.emitter_enabled:=1 --params-file /tmp/launch_params_4_hoy6sq -r rgb/image:=/rs/rgb_image_sync -r rgb/camera_info:=/rs/rgb_camera_info_sync -r depth/image:=/rs/aligned_depth_image_sync']. [rtabmap_viz-4] [WARN] [1741163153.488268078] [rtabmap_viz]: rtabmap_viz: Did not receive data since 5 seconds! Make sure the input topics are published ("$ ros2 topic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). Ajusting topic_queue_size (10) and sync_queue_size (10) can also help for better synchronization if framerates and/or delays are different. If topics are not published at the same rate, you could increase "sync_queue_size" and/or "topic_queue_size" parameters (current=10 and 10 respectively).
We tested the same rtabmap_ros package on a server with Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz, and it worked without issues. However, we are facing these problems on AGX Orin.
What could be the problem? Please guide us to resolve this issue.
I haven't tried multiple cameras yet but installing from source with the direction on ros2 branch works for me on agx orin humble jetpack 6.2 with zed camera. Did you do the ros2 topic hz to make sure topics are actually published? Have you tested that the camera works with the jetson with normal drivers besides using rtabmap
This:
[rgbd_odometry-2] terminate called after throwing an instance of 'cv::Exception'
[rgbd_odometry-2] what(): OpenCV(4.10.0-dev) /home/orin-5/opencv/modules/core/src/alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 55505507871600 bytes in function 'OutOfMemoryError'
[rgbd_odometry-2]
looks like a incompatibility between cv_bridge and rtabmap libraries. rtabmap_ros will link on both cv_bridge and rtabmap libraries, if you installed opencv from source and linked rtabmap with it, you have to remove cv_bridge binaries and rebuild it in your ros2 workspace at the same time than rtabmap_ros. This will make sure that rtabmap_ros links indirectly to same opencv libraries. Other option not involving the removal of all binary packages depending on cv_bridge is to build the same opencv version than the system binaries, or don't use opencv built from source at all if possible.
