rclcpp icon indicating copy to clipboard operation
rclcpp copied to clipboard

Events Executor crash in composition

Open jplapp opened this issue 1 year ago • 1 comments

Bug report

Required Info:

  • Operating System: Ubuntu 18
  • Installation type: source
  • Version or commit hash: iron
  • DDS implementation: cyclone
  • Client library (if applicable): rclcpp

Steps to reproduce issue

  1. launch foxglove bridge and nav2 stack in a composed node container, using an isolated events executor for each container
  2. disconnect from foxglove bridge -> it closes all subscribers

Expected behavior

nav2 keeps running

Actual behavior

container crashes. My rough guess is that some message wants to be delivered, but the subscriber went away and so it cannot be and runs into a segfault Screenshot from 2024-01-09 15-18-20

I've tried to narrow this done to a smaller, reproducible example but without luck so far. Maybe someone can point me in a direction here.

jplapp avatar Jan 10 '24 09:01 jplapp

Looking at the stack traces here, I'm thinking that it's probably something in nav2_controller/libcontroller_server_core since that's the bottom of the stack.

When any single node crashes, it's going to bring down the entire container, this is the trade off to the performance gains of being in the same process. To debug further, I would recommend moving your individual nodes outside of the container, starting with the nav2_controller node. You can then run that individual node under gdb to find out what is happening a little more.

Another useful diagnostic would be to know if this happens for just the events executor or all of the executor implementations.

mjcarroll avatar Jan 10 '24 13:01 mjcarroll