rtabmap icon indicating copy to clipboard operation
rtabmap copied to clipboard

Support more image types (depth/disparity confidence, thermal, segmentation) registered to RGB/Depth camera

Open matlabbe opened this issue 1 year ago • 1 comments

At the same time, we may revise how we differentiate between right and depth images in the database (they are saved under same field, only their type differentiate them): https://github.com/introlab/rtabmap/blob/465ea85d91d8340ccc961f5997333e7e16c8258a/corelib/include/rtabmap/core/SensorData.h#L328-L329

Constraints:

  1. Multi-cameras requires that all images across all cameras have same resolution
  2. RGB images are stitched together (currently simplifying features extraction), though that could be changed (if so, that could remove constraint 1)
  3. ...

That could help to support new workflows with RTAB-Map:

  • (Thermal image) https://answers.ros.org/question/211785/how-to-generate-thermal-colored-map
  • (Segmented image from CNN) https://github.com/introlab/rtabmap_ros/issues/317#issuecomment-504038533
  • (Depth confidence image) http://official-rtab-map-forum.206.s1.nabble.com/Noisy-floor-with-ARKit-tp10530p10541.html
  • (iOS missing surfaces) http://official-rtab-map-forum.206.s1.nabble.com/RTAB-vs-other-scanning-apps-for-iOS-td10615.html#a10703

Downstream changes to support this new feature:

  • Revise all post-processing features using RGB and or depth images (RTAB-Map standalone, rtabmap-databaseViewer, rtabmap-export)
  • Point cloud generation functions would need to be updated
  • ROS1 and ROS2 wrappers would need to be updated
  • Android/iOS code would need to be updated
  • Add new ways to visualize all image types (e..g, ImageView)
  • Add new options to generate point cloud data depending on the image layers we want and their effect. For example: I want to generate point cloud from only high confidence depth values and apply segmented class ID as color on it.

matlabbe avatar Oct 13 '24 19:10 matlabbe

Another related post: http://official-rtab-map-forum.206.s1.nabble.com/Bunlder-export-what-poses-td11037.html#a11213

matlabbe avatar May 29 '25 18:05 matlabbe