ros2cli icon indicating copy to clipboard operation
ros2cli copied to clipboard

NodeStrategy init Hanging

Open gcullie opened this issue 4 years ago • 18 comments
trafficstars

https://github.com/ros2/ros2cli/blob/094bcd9d8ba0a381f3598eeaba03eb45ed3d2df1/ros2cli/ros2cli/node/strategy.py#L25

Hangs within this method. Issue: #ros2/ros2/issues/1093

can recreate with command: ros2 topic list (hangs with constant increase of memory allocation until manually killed)

and directly recreate with:

/root/wsdev/src/jtbot/src/topicList.py 
lv = ListVerb()
v.main(args=None)

Environment:

ros-foxy-desktop/focal,now 0.9.2-1focal.20210226.004308 amd64 [installed]
Docker ubuntu 20.04 
python3.8/focal-updates,focal-security,now 3.8.5-1~20.04.2 amd64 [installed,automatic]

gcullie avatar Mar 15 '21 17:03 gcullie

@gcullie thank you for opening issue!

so far, i was unable to reproduce this issue...

  • amd64
    • AMD Ryzen 9 3900X 12-Core Processor
    • ros-foxy-ros-base: 0.9.2-1focal.20210226.000028
    • while true; do ros2 topic list; done does not hang for 30mins
  • aarch64
    • Cortex-A72 (Raspi4)
    • ros-foxy-ros-base: 0.9.2-1focal.20210226.033419
    • while true; do ros2 topic list; done does not hang for 30mins

if you find anything that i can try in my environment, that would be really appreciated :exclamation:

fujitatomoya avatar Mar 16 '21 04:03 fujitatomoya

I am having a similar issue too where 'ros2 topic list' and 'ros2 node list' hang. I am also running in a docker image on Ubuntu 20.04. It seems it finally works maybe after rebooting the host (at least I did that and it started working).

ros-foxy-ros-base version: 0.9.2-1focal.20210106.200442

bijoua29 avatar Mar 18 '21 16:03 bijoua29

I'm also not able to reproduce this in a Foxy docker container.

Any other suggestions on reproducing this?

audrow avatar Mar 19 '21 16:03 audrow

I'm using osrf/ros:foxy-desktop as the base image for my docker. The issue is random and I can't really detect a pattern to it yet. It happens when I run my application and then I want to list topics and nodes. Closing the application doesn't seem to help. When it happens, only a reboot of the host seems to fix it. Anyway, I will keep trying to get more information on the failure mode.

bijoua29 avatar Mar 19 '21 16:03 bijoua29

I'll give it a try on the same docker image as you. I've been using ros:foxy.

Edit: osrf/ros:foxy-desktop also worked fine for me.

audrow avatar Mar 19 '21 16:03 audrow

Yeah. All I can say is that it is not very repeatable but it happens about once or twice a day for me in repeated use. Will keep trying to gather more information. If there is something you want me to look at when this happens, let me know.

bijoua29 avatar Mar 19 '21 18:03 bijoua29

I've just experienced this issue on a Ubuntu 20.04 AMD64 system running running ROS2 galactic. The problem resolved itself after rebooting the system.

mschuckmann avatar Dec 15 '21 19:12 mschuckmann

I've just experienced this issue on a Ubuntu 20.04 AMD64 system running running ROS2 galactic. The problem resolved itself after rebooting the system.

This was the same resolution for me as well i.e. rebooting. I will say that I haven't experienced this problem in a while. It is also the case that I have been running 'rolling' on Ubuntu 20.04 for the past 3-4 months so that might have fixed the problem although I can't say for sure. Previously, I was using foxy on Ubuntu 20.04.

bijoua29 avatar Dec 15 '21 19:12 bijoua29

I will say that I experienced the problem after some failed attempts to run my launch file so it could be that there was some hung process or zombies that were confusing ros2.

mschuckmann avatar Dec 15 '21 19:12 mschuckmann

Did someone solve the problem? I have ros2 rolling on a Ubuntu 20.04 and it hangs too, but it is not solved via reboot.

konstihengge avatar Jul 12 '22 14:07 konstihengge

Did someone solve the problem? I have ros2 rolling on a Ubuntu 20.04 and it hangs too, but it is not solved via reboot.

Rolling isn't well supported on 20.04. It's currently listed an a tier 3 platform (can be built from source, but not well tested by us). https://ros.org/reps/rep-2000.html#humble-hawksbill-may-2022-may-2027

You may want to try using Ubuntu 22.04 or using Galactic or Foxy (LTS) on 20.04.

but it is not solved via reboot.

That's strange to me. If it doesn't go away on Galactic or Foxy, maybe post more details about your setup.

audrow avatar Jul 12 '22 18:07 audrow

Thank you for your advice @audrow I'm now using Humble with Ubuntu 22.04 and it's working fine

konstihengge avatar Jul 22 '22 07:07 konstihengge

I'm still having the same issue on my system.

Ubuntu Version 22.04
Linux austin-greisman 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
ros-humble-ros2cli/jammy,now 0.18.4-1jammy.20221207.031440 amd64 [installed,automatic]
ros-humble-ros2controlcli/jammy 2.18.0-1jammy.20221207.043415 amd64

ros-humble-cyclonedds-dbgsym/jammy 0.9.1-1jammy.20221108.154039 amd64
ros-humble-cyclonedds/jammy,now 0.9.1-1jammy.20221108.154039 amd64 [installed,automatic]

It seems the only temporary fix is restarting my system. ros2 daemon stop && ros2 daemon start also hangs...

I first noticed this issue while I was debugging C++ code. It seemed to be causing issues with Cyclone

austin-InDro avatar Jan 06 '23 16:01 austin-InDro

I have the same issues, indeed using Cyclone too.

stijnb1234 avatar Jun 12 '23 11:06 stijnb1234

Hi,

Same issue here. The only solution I found so far is to reboot.

pucciland95 avatar Jun 14 '23 09:06 pucciland95

Had this same issue when trying to run ROS2 Humble through a Ubuntu 22.04 Docker container on an Arch Linux setup. Rerunning/rebuilding the container didn't fix the issue for me either, but the solution that worked for me was to run the command ros2 topic list or ros2 node list with the --no-daemon flag.

Haven't yet tested with any other ROS distributions, but this solution should work there too.

arjunchainani avatar Apr 16 '24 00:04 arjunchainani

@arjunchainani out of curiosity, do you also use cyclonedds?

fujitatomoya avatar Apr 16 '24 01:04 fujitatomoya

@fujitatomoya I was previously just using the default Fast DDS, but I just tried Cyclone and the fix seems to work with Cyclone as well.

arjunchainani avatar Apr 16 '24 01:04 arjunchainani