isaac_ros_freespace_segmentation
isaac_ros_freespace_segmentation copied to clipboard
Hardware-accelerated, deep-learned freespace segmentation
Isaac ROS Freespace Segmentation
Hardware-accelerated, deep-learned freespace segmentation
Overview
Isaac ROS Freespace Segmentation contains an ROS 2 package to produce occupancy grids for navigation. By processing a freespace segmentation mask with the pose of the robot relative to the ground, Bi3D Freespace produces an occupancy grid for Nav2, which is used to avoid obstacles during navigation. This package is GPU accelerated to provide real-time, low latency results in a robotics application. Bi3D Freespace provides an additional occupancy grid source for mobile robots (ground based).
isaac_ros_bi3d
is used in a graph of nodes to provide a freespace
segmentation mask as one output from a time-synchronized input left and
right stereo image pair. The freespace mask is used by
isaac_ros_bi3d_freespace
with TF pose of the camera relative to the
ground to compute planar freespace into an occupancy grid as input to
Nav2.
There are multiple methods to predict the occupancy grid as an input to navigation. None of these methods are perfect; each has limitations on the accuracy of its estimate from the sensor providing measured observations. Each sensor has a unique field of view, range to provide its measured view of the world, and corresponding areas it does not measure. Bi3D Freespace provides a diverse approach to identifying obstacles from freespace. Stereo camera input used for this function is diverse relative to lidar, and has a better vertical field of view than most lidar units, allowing for perception of low lying obstacles that lidar can miss. Bi3D Freespace provides a robust, vision-based complement to lidar occupancy scanning.
Isaac ROS NITROS Acceleration
This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.
Performance
Sample Graph |
Input Size |
AGX Orin |
Orin NX |
Orin Nano 8GB |
x86_64 w/ RTX 4060 Ti |
---|---|---|---|---|---|
Freespace Segmentation Node |
576p |
1760 fps 1.2 ms |
1410 fps 1.6 ms |
1060 fps 2.3 ms |
3500 fps 0.32 ms |
Freespace Segmentation Graph |
576p |
45.9 fps 41 ms |
27.6 fps 95 ms |
21.3 fps 110 ms |
91.0 fps 30 ms |
Documentation
Please visit the Isaac ROS Documentation to learn how to use this repository.
Packages
Latest
Update 2023-10-18: Updated for Isaac ROS 2.0.0.