geometry2 icon indicating copy to clipboard operation
geometry2 copied to clipboard

Fix invalid timer handle exception

Open z80020100 opened this issue 3 years ago • 5 comments

  • This PR try to fix invalid timer handle exception caused by removing handle repeatedly.
  • This issue is found when using Nav2 and set a small transform_tolerance for local_costmap and global_costmap.
  • Specific error messages are as follows:
    [server-9] [planner_server-4] terminate called after throwing an instance of 'tf2_ros::InvalidTimerHandleException'
    [server-9] [planner_server-4]   what():  Invalid timer handle in remove()
    
  • We found this exception is thrown from https://github.com/ros2/geometry2/blob/16562cee8694c11f1f82b2bdbde2814fca1c7954/tf2_ros/src/create_timer_ros.cpp#L93
  • If adding more messages for debug, we found that there is a high probability that handle will be removed repeatedly.
    [server-9] [planner_server-4] [INFO] [1636089969.121297887] [tf2_buffer]: [cliff] remove@217, it->first=286, this->timer_interface_=0x55eee89ff140
    [server-9] [planner_server-4] [INFO] [1636089969.126648738] [tf2_buffer]: [cliff] remove@286, timer_handle=286, timer_interface_=0x55eee89ff140
    [server-9] [planner_server-4] terminate called after throwing an instance of 'tf2_ros::InvalidTimerHandleException'
    [server-9] [planner_server-4]   what():  Invalid timer handle in remove()
    
  • Therefore, we have made a change to ensure that timer_handle only can be removed when it can be found in timer_to_request_map_

z80020100 avatar Nov 05 '21 23:11 z80020100

How should I add a test case?

z80020100 avatar Nov 06 '21 02:11 z80020100

How should I add a test case?

Can you write a code snippet that uses the tf2_ros interface in a way that throws this timer handle exception without the patch (but not with it)? If so, that snippet ~~should~~ may live as a test case in tf2_ros/test/test_buffer.cpp.

aprotyas avatar Nov 06 '21 09:11 aprotyas

Thank you for the feedback. I will try to write a test case and push it later.

z80020100 avatar Nov 08 '21 01:11 z80020100

@z80020100 there are linter errors in the Rpr job, can you address them please? https://build.ros2.org/job/Rpr__geometry2__ubuntu_focal_amd64/372/testReport/

aprotyas avatar Nov 09 '21 07:11 aprotyas

@aprotyas Sure, I will fix them.

z80020100 avatar Nov 09 '21 07:11 z80020100

status: works for me :+1:

cmraaron avatar Mar 01 '23 13:03 cmraaron

Will this be merged into humble? @aprotyas I'm sure other people will also face this issue (we did and we independently identified the same bug). If this is good, then we should merge this into the code base or at least mark it as an issue so that others can find this.

swarajpeppermint avatar Apr 28 '23 14:04 swarajpeppermint

@swarajpeppermint thanks for flagging this. Since I’m not a maintainer I can’t actually merge the code, but I’m tagging @clalancette who can hopefully get this PR across the finish line

aprotyas avatar Apr 28 '23 16:04 aprotyas

CI:

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Windows Build Status

clalancette avatar Jul 20 '23 14:07 clalancette

fantastic!, can we backport to iron and humble?

cmraaron avatar Jul 21 '23 03:07 cmraaron

https://github.com/Mergifyio backport iron humble

ahcorde avatar Jul 21 '23 07:07 ahcorde

backport iron humble

✅ Backports have been created

mergify[bot] avatar Jul 21 '23 07:07 mergify[bot]

This issue was independently reported to Nav2, ROS answers and Robotics Stack Exchange and I couldn't find a fix for a long time. Thanks for this fix, it was really a frequent problem for our production robots.

at least mark it as an issue

In retrospect, this would have probably been a good idea to help find this issue/PR more easily

tonynajjar avatar Aug 29 '23 09:08 tonynajjar

@z80020100 @swarajpeppermint @clalancette Can this be backported to Galactic or Foxy ?

anath93 avatar Jan 18 '24 15:01 anath93

@anath93 Galactic and Foxy are EOL we are not merging more PRs in these branches.

ahcorde avatar Jan 18 '24 16:01 ahcorde

@ahcorde these are the only LTS version on 20.04, there is still a lot of community and would be helpful for the community. Can this be considered still ?

anath93 avatar Jan 18 '24 16:01 anath93

Can this be considered still ?

No, sorry. We have no way to deliver fixes for those distributions anymore.

clalancette avatar Jan 18 '24 16:01 clalancette