Isaac ROS Visual SLAM

Overview

This repository provides a ROS2 package that performs stereo visual simultaneous localization and mapping (VSLAM) and estimates stereo visual inertial odometry using the Isaac Elbrus GPU-accelerated library. It takes in a time-synced pair of stereo images (grayscale) along with respective camera intrinsics to publish the current pose of the camera relative to its start pose.

Elbrus is based on two core technologies: Visual Odometry (VO) and Simultaneous Localization and Mapping (SLAM).

Visual SLAM is a method for estimating a camera position relative to its start position. This method has an iterative nature. At each iteration, it considers two consequential input frames (stereo pairs). On both the frames, it finds a set of keypoints. Matching keypoints in these two sets gives the ability to estimate the transition and relative rotation of the camera between frames.

Simultaneous Localization and Mapping is a method built on top of the VO predictions. It aims to improve the quality of VO estimations by leveraging the knowledge of previously seen parts of a trajectory. It detects if the current scene was seen in the past (i.e. a loop in camera movement) and runs an additional optimization procedure to tune previously obtained poses.

Along with visual data, Elbrus can optionally use Inertial Measurement Unit (IMU) measurements. It automatically switches to IMU when VO is unable to estimate a pose; for example, when there is dark lighting or long solid featureless surfaces in front of a camera. Elbrus delivers real-time tracking performance: more than 60 FPS for VGA resolution. For the KITTI benchmark, the algorithm achieves a drift of ~1% in localization and an orientation error of 0.003 degrees per meter of motion. Elbrus allows for robust tracking in various environments and with different use cases: indoor, outdoor, aerial, HMD, automotive, and robotics.

To learn more about Elbrus SLAM click here.

Table of Contents

• Isaac ROS Visual SLAM
  • Overview
  • Table of Contents
  • Latest Update
  • Supported Platforms
    • Docker
  • Coordinate Frames
  • Quickstart
  • Next Steps
    • Try More Examples
    • Customize your Dev Environment
  • Package Reference
    • isaac_ros_visual_slam
      • Usage
      • ROS Parameters
      • ROS Topics Subscribed
      • ROS Topics Published
      • ROS Services Advertised
      • ROS Actions Advertised
  • Troubleshooting
    • Isaac ROS Troubleshooting
    • Troubleshooting Suggestions
  • Updates

Latest Update

Update 2022-06-30: Support for ROS2 Humble

Supported Platforms

This package is designed and tested to be compatible with ROS2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.

Docker

To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.

Note: All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.

Coordinate Frames

This section describes the coordinate frames that are involved in the VisualSlamNode. The frames discussed below are oriented as follows:

1. input_base_frame: The name of the frame used to calculate transformation between baselink and left camera. The default value is empty (''), which means the value of base_frame_ will be used. If input_base_frame_ and base_frame_ are both empty, the left camera is assumed to be in the robot's center.
2. input_left_camera_frame: The frame associated with left eye of the stereo camera. Note that this is not the same as the optical frame. The default value is empty (''), which means the left camera is in the robot's center and left_pose_right will be calculated from the CameraInfo message.
3. input_right_camera_frame: The frame associated with the right eye of the stereo camera. Note that this is not the same as the optical frame. The default value is empty (''), which means left and right cameras have identity rotation and are horizontally aligned, so left_pose_right will be calculated from CameraInfo.
4. input_imu_frame: The frame associated with the IMU sensor (if available). It is used to calculate left_pose_imu.

Quickstart

1. Set up your development environment by following the instructions here.

2. Clone this repository and its dependencies under ~/workspaces/isaac_ros-dev/src.

   cd ~/workspaces/isaac_ros-dev/src
   

   git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_visual_slam
   

   git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common
   

3. Pull down a ROS Bag of sample data:

   cd ~/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam && \ 
     git lfs pull -X "" -I isaac_ros_visual_slam/test/test_cases/rosbags/
   

4. Launch the Docker container using the run_dev.sh script:

   cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
     ./scripts/run_dev.sh
   

5. Inside the container, build and source the workspace:

   cd /workspaces/isaac_ros-dev && \
     colcon build --symlink-install && \
     source install/setup.bash
   

6. (Optional) Run tests to verify complete and correct installation:

   colcon test --executor sequential
   

7. Run the following launch files in the current terminal:

   ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam.launch.py
   

8. In a second terminal inside the Docker container, prepare rviz to display the output:

   source /workspaces/isaac_ros-dev/install/setup.bash && \
     rviz2 -d src/isaac_ros_visual_slam/isaac_ros_visual_slam/rviz/default.cfg.rviz 
   

9. In an another terminal inside the Docker container, run the following ros bag file to start the demo:

   source /workspaces/isaac_ros-dev/install/setup.bash && \
     ros2 bag play src/isaac_ros_visual_slam/isaac_ros_visual_slam/test/test_cases/rosbags/small_pol_test/
   

   Rviz should start displaying the point clouds and poses like below:

Next Steps

Try More Examples

To continue your exploration, check out the following suggested examples:

• Tutorial with Isaac Sim

Customize your Dev Environment

To customize your development environment, reference this guide.

Package Reference

isaac_ros_visual_slam

Usage

ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam.launch.py

ROS Parameters

ROS Topics Subscribed

ROS Topics Published

ROS Services Advertised

ROS Actions Advertised

Troubleshooting

Isaac ROS Troubleshooting

For solutions to problems with Isaac ROS, please check here.

Troubleshooting Suggestions

• If RViz is not showing the poses, check the Fixed Frame value.
• If you are seeing Tracker is lost. messages frequently, it could be caused by the following issues:
  • Fast motion causing the motion blur in the frames.
  • Low-lighting conditions.
  • The wrong camerainfo is being published.
• For better performance:
  • Increase the capture framerate from the camera to yield a better tracking result.
  • If input images are noisy, you can use the denoise_input_images flag in the node.

Updates