Draft: Semantic segmentation plugin for `nav2_costmap_2d`

[pepisg]

---

Basic Info

Info	Please fill out this column
Ticket(s) this addresses	None to my knowledge
Primary OS tested on	Ubuntu
Robotic platform tested on	Custom skid steer drive robot on gazebo and in real life

---

Description of contribution in a few bullet points

• I added a semantic segmentation plugin to the nav2_costmap_2d package for adding segmentation data to costmaps. For now this assumes segmentation comes from and RGBD sensor that can produce aligned pointclouds. The configuration for this plugin looks like this:

    semantic_segmentation_layer:
        plugin: nav2_costmap_2d::SemanticSegmentationLayer
        enabled: True
        segmentation_topic: "/semantic_segmentation/segmentation/image_raw"
        pointcloud_topic: "/camera/depth/color/points"
        max_obstacle_distance: 3.5
        min_obstacle_distance: 0.5
        class_types: ["traversable", "warning", "danger"]
        traversable:
          classes: ["Sidewalk"]
          cost: 0
        warning:
          classes: ["Pads"]
          cost: 90
        danger:
          classes: ["Grass", "Other"]
          cost: 254

A ray tracing approach was tried to make the plugin work with calibrated RGB cameras but was discarded because projections of obstacles over the ground plane caused obstacles to extend beyond their actual location. @SteveMacenski let me know if you think more information about why this approach was not pursued should be added.

The plugin is dependant on new segmentation messages. @SteveMacenski, let me know if you agree with this definition for opening a PR to vision_msgs

Description of documentation updates required from your changes

• Added a new costmap plugin, so a configuration guide needs to be added. The documentation on nav2_costmap_2d also needs updating
• Also it would be good to add a tutorial on navigation2_tutorials. @SteveMacenski I could provide a demo segmentation node for detecting gazebo's ground plane on an image, let me know what you think would be the best way of doing this.

PD: I will start doing this after the first review in which I expect several changes will be requested

Future work that may be required in bullet points

• Extend the plugin to work with any combination of segmentation sources and pointclouds (e.x: 3D lidar + RGB camera)
• Integrate various handling methods for adjusting cost of pixels according to the confidence of the inference

For Maintainers:

• [ ] Check that any new parameters added are updated in navigation.ros.org
• [ ] Check that any significant change is added to the migration guide
• [ ] Check that any new features OR changes to existing behaviors are reflected in the tuning guide
• [ ] Check that any new functions have Doxygen added
• [ ] Check that any new features have test coverage
• [ ] Check that any new plugins is added to the plugins page
• [ ] If BT Node, Additionally: add to BT's XML index of nodes for groot, BT package's readme table, and BT library lists

NOTE: This PR will not build due to its dependency to vision_msgs's new messages

[mergify[bot]]

@pepisg, your PR has failed to build. Please check CI outputs and resolve issues. You may need to rebase or pull in main due to API changes (or your contribution genuinely fails).

[SteveMacenski]

Please open the PR with vision_msgs + tag me and we'll discuss there

[pepisg]

Please open the PR with vision_msgs + tag me and we'll discuss there

Just opened the PR on vision messages