Autoware keeps following old local path after `behavior_velocity_planner` is dead

[VRichardJP]

Checklist

• [X] I've read the contribution guidelines.
• [X] I've searched other issues and no duplicate issues were found.
• [X] I'm convinced that this is not my fault but a bug.

Description

While I experienced the behavior_velocity_planner crash (described here: https://github.com/autowarefoundation/autoware.universe/issues/1247), I observed autoware did not make the vehicle stop immediately after the crash, but kept driving along the last generated local path, reached the end of the local path and kept driving straight after. Autoware only triggered an emergency stop when the vehicle was ~3m beyond that last point (out of path detected).

Although the out of path detection range could be reduced to have the emergency triggered earlier, it is very unsafe for autoware to keep following an old local path. For example, even though some new obstacle would be detected on the road, autoware would simply ignore them and follow the last local path it has received.

Expected behavior

An emergency should be triggered if local paths are not generated anymore

Actual behavior

Autoware keep driving on the old path, even after its end has been reached.

Steps to reproduce

1. run autoware on a vehicle
2. kill behavior_velocity_planner while in autonomous mode

Versions

• Ubuntu 20.04
• galactic
• main

Possible causes

No response

Additional context

No response

[xmfcx]

A node monitoring node should check the critical nodes and issue emergency stop if one of them crashes.

You can try to add topic_state_monitor to this launch file: https://github.com/autowarefoundation/autoware.universe/blob/main/launch/tier4_system_launch/launch/system.launch.xml

And somehow it should issue emergency message and stop the vehicle through the emergency_handler

[mitsudome-r]

@mitsudome-r to provide additional documentation regarding emergency handler

[mitsudome-r]

I've started working on this. I think I will be able to create PR this week.

[mitsudome-r]

I figured out that with the default configuration, topic monitoring function is not turned on. (This probably is because it is not always safe to make a sudden stop at node failure)

If you add use_emergency handling to control module, you should be able to turn on this function and make vehicle stop when ever major planning nodes dies with unknown error.

https://github.com/autowarefoundation/autoware_launch/blob/515239b9f1d9c4161629f25c1b8e4c6ec62acfdd/autoware_launch/launch/autoware.launch.xml#L117

Modify the above launch as the following:

  <group if="$(var launch_control)">
    <include file="$(find-pkg-share tier4_control_launch)/launch/control.launch.py">
      <arg name="lateral_controller_mode" value="mpc_follower"/>
      <arg name="vehicle_info_param_file" value="$(find-pkg-share $(var vehicle_model)_description)/config/vehicle_info.param.yaml"/>
      <arg name="use_emergency_handling" value="true"/>
    </include>
  </group>

[mitsudome-r]

I will close this issue once I add the above explanation in Autoware Documentation

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity.

[xmfcx]

@mitsudome-r is working on https://github.com/autowarefoundation/autoware-documentation/pull/278

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity.

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity.

[VRichardJP]

I have not seen the motion planner dying while autoware is driving for a while, so I am not sure if the problem still exists.

I think now there would be an emergency stop?

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity.