depthai-ros icon indicating copy to clipboard operation
depthai-ros copied to clipboard

Published 20 hours ago • verified publisher icon luxonis

camera.launch: IMU Data is (sometimes) not being published

Open everskies opened this issue 2 years ago • 4 comments

Describe the bug I've been trying to get the camera.launch.py example to publish IMU data together with the stereo data and RGB image. The /oak/imu/data topic gets created:

ros2 topic list
/joint_states
/oak/imu/data
/oak/rgb/camera_info
/oak/rgb/image_raw
/oak/rgb/image_raw/compressed
/oak/rgb/image_raw/compressedDepth
/oak/rgb/image_raw/theora
/oak/stereo/camera_info
/oak/stereo/image_raw
/oak/stereo/image_raw/compressed
/oak/stereo/image_raw/compressedDepth
/oak/stereo/image_raw/theora
/parameter_events
/robot_description
/rosout
/tf
/tf_static

However, no data seems to arrive:

ros2 topic hz /oak/imu/data
WARNING: topic [/oak/imu/data] does not appear to be published yet

I've tried both the current humble release and one built from scratch.

Minimal Reproducible Example ros2 launch depthai_ros_driver camera.launch.py

[INFO] [launch]: All log files can be found below /home/sleziak/.ros/log/2023-04-18-22-47-30-727616-bubble-2551
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [robot_state_publisher-1]: process started with pid [2562]
[INFO] [component_container-2]: process started with pid [2564]
[component_container-2] [INFO] [1681850852.374532197] [oak_container]: Load Library: /home/sleziak/ros/install/depthai_ros_driver/lib/libdepthai_ros_driver.so
[robot_state_publisher-1] [INFO] [1681850852.540030735] [oak_state_publisher]: got segment oak
[robot_state_publisher-1] [INFO] [1681850852.540405935] [oak_state_publisher]: got segment oak-d-base-frame
[robot_state_publisher-1] [INFO] [1681850852.540491657] [oak_state_publisher]: got segment oak_imu_frame
[robot_state_publisher-1] [INFO] [1681850852.540537342] [oak_state_publisher]: got segment oak_left_camera_frame
[robot_state_publisher-1] [INFO] [1681850852.540575471] [oak_state_publisher]: got segment oak_left_camera_optical_frame
[robot_state_publisher-1] [INFO] [1681850852.540611785] [oak_state_publisher]: got segment oak_model_origin
[robot_state_publisher-1] [INFO] [1681850852.540646452] [oak_state_publisher]: got segment oak_rgb_camera_frame
[robot_state_publisher-1] [INFO] [1681850852.540679822] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame
[robot_state_publisher-1] [INFO] [1681850852.540713359] [oak_state_publisher]: got segment oak_right_camera_frame
[robot_state_publisher-1] [INFO] [1681850852.540746488] [oak_state_publisher]: got segment oak_right_camera_optical_frame
[component_container-2] [INFO] [1681850852.691349716] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1681850852.691494308] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1681850852.745608206] [oak]: No ip/mxid specified, connecting to the next available device.
[component_container-2] [INFO] [1681850855.725776186] [oak]: Camera with MXID: 18443010F1213B1300 and Name: 1.1.2 connected!
[component_container-2] [INFO] [1681850855.727304153] [oak]: USB SPEED: SUPER
[component_container-2] [INFO] [1681850855.750146502] [oak]: Device type: OAK-D-PRO-AF
[component_container-2] [INFO] [1681850855.757216437] [oak]: Pipeline type: RGBD
[component_container-2] [INFO] [1681850855.801349463] [oak]: NN Family: mobilenet
[component_container-2] [INFO] [1681850856.071826360] [oak]: NN input size: 300 x 300. Resizing input image in case of different dimensions.
[component_container-2] [INFO] [1681850856.277071831] [oak]: Finished setting up pipeline.
[component_container-2] [INFO] [1681850857.683827872] [oak]: Camera ready!
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak' in container '/oak_container'

Expected behavior There should be data being published to the /oak/imu/data topic

log_system_information.json 
{
    "architecture": "64bit ELF",
    "machine": "aarch64",
    "platform": "Linux-5.15.0-1026-raspi-aarch64-with-glibc2.35",
    "processor": "aarch64",
    "python_build": "main Mar 10 2023 10:55:28",
    "python_compiler": "GCC 11.3.0",
    "python_implementation": "CPython",
    "python_version": "3.10.6",
    "release": "5.15.0-1026-raspi",
    "system": "Linux",
    "version": "#28-Ubuntu SMP PREEMPT Fri Mar 10 14:28:52 UTC 2023",
    "win32_ver": "",
    "packages": [
        "action-msgs==1.2.1",
        "action-tutorials-interfaces==0.20.3",
        "action-tutorials-py==0.20.3",
        "actionlib-msgs==4.2.3",
        "ament-cmake-test==1.3.3",
        "ament-copyright==0.12.5",
        "ament-cppcheck==0.12.5",
        "ament-cpplint==0.12.5",
        "ament-flake8==0.12.5",
        "ament-index-python==1.4.0",
        "ament-lint==0.12.5",
        "ament-lint-cmake==0.12.5",
        "ament-package==0.14.0",
        "ament-pep257==0.12.5",
        "ament-uncrustify==0.12.5",
        "ament-xmllint==0.12.5",
        "angles==1.15.0",
        "appdirs==1.4.4",
        "argcomplete==1.8.1",
        "attrs==21.2.0",
        "Automat==20.2.0",
        "Babel==2.8.0",
        "bcrypt==3.2.0",
        "beautifulsoup4==4.10.0",
        "beniget==0.4.1",
        "blinker==1.4",
        "bond==3.0.2",
        "Brotli==1.0.9",
        "builtin-interfaces==1.2.1",
        "camera-calibration==3.0.0",
        "catkin-pkg==0.5.2",
        "catkin-pkg-modules==0.5.2",
        "certifi==2020.6.20",
        "chardet==4.0.0",
        "charset-normalizer==3.1.0",
        "click==8.0.3",
        "cloud-init==23.1.1",
        "colcon-argcomplete==0.3.3",
        "colcon-bash==0.4.2",
        "colcon-cd==0.1.1",
        "colcon-cmake==0.2.27",
        "colcon-common-extensions==0.3.0",
        "colcon-core==0.12.1",
        "colcon-defaults==0.2.8",
        "colcon-devtools==0.2.3",
        "colcon-installed-package-information==0.1.0",
        "colcon-library-path==0.2.1",
        "colcon-metadata==0.2.5",
        "colcon-notification==0.2.15",
        "colcon-output==0.2.13",
        "colcon-override-check==0.0.1",
        "colcon-package-information==0.3.3",
        "colcon-package-selection==0.2.10",
        "colcon-parallel-executor==0.2.4",
        "colcon-pkg-config==0.1.0",
        "colcon-powershell==0.3.7",
        "colcon-python-setup-py==0.2.8",
        "colcon-recursive-crawl==0.2.1",
        "colcon-ros==0.3.23",
        "colcon-test-result==0.3.8",
        "colcon-zsh==0.4.0",
        "colorama==0.4.4",
        "command-not-found==0.3",
        "composition-interfaces==1.2.1",
        "configobj==5.0.6",
        "constantly==15.1.0",
        "cov-core==1.15.0",
        "coverage==6.2",
        "cryptography==3.4.8",
        "cv-bridge==3.2.1",
        "cycler==0.11.0",
        "dbus-python==1.2.18",
        "decorator==4.4.2",
        "demo-nodes-py==0.20.3",
        "depthai==2.21.2.0",
        "depthai-ros-driver==2.7.1",
        "depthai-ros-msgs==2.7.1",
        "diagnostic-msgs==4.2.3",
        "diagnostic-updater==3.1.2",
        "distlib==0.3.6",
        "distro==1.7.0",
        "distro-info===1.1build1",
        "docutils==0.17.1",
        "domain-coordinator==0.10.0",
        "dwb-msgs==1.1.6",
        "empy==3.3.4",
        "example-interfaces==0.9.3",
        "examples-rclpy-executors==0.15.1",
        "examples-rclpy-minimal-action-client==0.15.1",
        "examples-rclpy-minimal-action-server==0.15.1",
        "examples-rclpy-minimal-client==0.15.1",
        "examples-rclpy-minimal-publisher==0.15.1",
        "examples-rclpy-minimal-service==0.15.1",
        "examples-rclpy-minimal-subscriber==0.15.1",
        "filelock==3.9.0",
        "flake8==4.0.1",
        "fonttools==4.29.1",
        "foxglove-msgs==2.1.1",
        "fs==2.4.12",
        "gast==0.5.2",
        "geographic-msgs==1.0.4",
        "geometry-msgs==4.2.3",
        "html5lib==1.1",
        "httplib2==0.20.2",
        "hyperlink==21.0.0",
        "idna==3.3",
        "image-geometry==3.2.1",
        "importlib-metadata==4.6.4",
        "incremental==21.3.0",
        "iniconfig==1.1.1",
        "interactive-markers==2.3.2",
        "jeepney==0.7.1",
        "Jinja2==3.0.3",
        "joint-state-publisher==2.3.0",
        "jsonpatch==1.32",
        "jsonpointer==2.0",
        "jsonschema==3.2.0",
        "keyring==23.5.0",
        "kiwisolver==1.3.2",
        "lark==1.1.1",
        "laser-geometry==2.4.0",
        "launch==1.0.4",
        "launch-ros==0.19.4",
        "launch-testing==1.0.4",
        "launch-testing-ros==0.19.4",
        "launch-xml==1.0.4",
        "launch-yaml==1.0.4",
        "launchpadlib==1.10.16",
        "lazr.restfulclient==0.14.4",
        "lazr.uri==1.0.6",
        "lifecycle-msgs==1.2.1",
        "logging-demo==0.20.3",
        "lxml==4.8.0",
        "lz4==3.1.3+dfsg",
        "map-msgs==2.1.0",
        "MarkupSafe==2.0.1",
        "matplotlib==3.5.1",
        "mccabe==0.6.1",
        "message-filters==4.3.2",
        "micro-ros-msgs==1.0.0",
        "more-itertools==8.10.0",
        "mpi4py==3.1.3",
        "mpmath==0.0.0",
        "nav-2d-msgs==1.1.6",
        "nav-msgs==4.2.3",
        "nav2-common==1.1.6",
        "nav2-msgs==1.1.6",
        "nav2-simple-commander==1.0.0",
        "netifaces==0.11.0",
        "nose2==0.9.2",
        "notify2==0.3",
        "numpy==1.21.5",
        "oauthlib==3.2.0",
        "olefile==0.46",
        "osrf-pycommon==2.0.2",
        "packaging==21.3",
        "pbr==5.11.1",
        "pcl-msgs==1.0.0",
        "pendulum-msgs==0.20.3",
        "pexpect==4.8.0",
        "Pillow==9.0.1",
        "pip==23.1",
        "platformdirs==3.1.1",
        "pluggy==0.13.0",
        "ply==3.11",
        "psutil==5.9.0",
        "ptyprocess==0.7.0",
        "py==1.10.0",
        "pyasn1==0.4.8",
        "pyasn1-modules==0.2.1",
        "pycairo==1.20.1",
        "pycodestyle==2.8.0",
        "pydocstyle==6.1.1",
        "pydot==1.4.2",
        "pyflakes==2.4.0",
        "Pygments==2.11.2",
        "PyGObject==3.42.1",
        "PyHamcrest==2.0.2",
        "PyJWT==2.3.0",
        "pyOpenSSL==21.0.0",
        "pyparsing==2.4.7",
        "PyQt5==5.15.6",
        "PyQt5-sip==12.9.1",
        "pyrsistent==0.18.1",
        "pyserial==3.5",
        "pytest==6.2.5",
        "pytest-cov==3.0.0",
        "python-apt==2.4.0+ubuntu1",
        "python-dateutil==2.8.1",
        "python-debian===0.1.43ubuntu1",
        "python-magic==0.4.24",
        "python-qt-binding==1.1.1",
        "pythran==0.10.0",
        "pytz==2022.1",
        "PyYAML==6.0",
        "qt-dotgraph==2.2.2",
        "qt-gui==2.2.2",
        "qt-gui-cpp==2.2.2",
        "qt-gui-py-common==2.2.2",
        "quality-of-service-demo-py==0.20.3",
        "rcl-interfaces==1.2.1",
        "rclpy==3.3.7",
        "rcutils==5.1.2",
        "requests==2.28.2",
        "resource-retriever==3.1.1",
        "rmw-dds-common==1.6.0",
        "robot-localization==3.3.1",
        "roman==3.3",
        "ros-gz-interfaces==0.244.9",
        "ros-ign-interfaces==0.244.9",
        "ros2action==0.18.5",
        "ros2bag==0.15.4",
        "ros2cli==0.18.5",
        "ros2component==0.18.5",
        "ros2doctor==0.18.5",
        "ros2interface==0.18.5",
        "ros2launch==0.19.4",
        "ros2lifecycle==0.18.5",
        "ros2multicast==0.18.5",
        "ros2node==0.18.5",
        "ros2param==0.18.5",
        "ros2pkg==0.18.5",
        "ros2run==0.18.5",
        "ros2service==0.18.5",
        "ros2topic==0.18.5",
        "rosbag2-interfaces==0.15.4",
        "rosbag2-py==0.15.4",
        "rosdep==0.22.1",
        "rosdep-modules==0.22.2",
        "rosdistro==0.9.0",
        "rosdistro-modules==0.9.0",
        "rosgraph-msgs==1.2.1",
        "rosidl-adapter==3.1.4",
        "rosidl-cli==3.1.4",
        "rosidl-cmake==3.1.4",
        "rosidl-generator-c==3.1.4",
        "rosidl-generator-cpp==3.1.4",
        "rosidl-generator-py==0.14.4",
        "rosidl-parser==3.1.4",
        "rosidl-runtime-py==0.9.3",
        "rosidl-typesupport-c==2.0.0",
        "rosidl-typesupport-cpp==2.0.0",
        "rosidl-typesupport-fastrtps-c==2.2.0",
        "rosidl-typesupport-fastrtps-cpp==2.2.0",
        "rosidl-typesupport-introspection-c==3.1.4",
        "rosidl-typesupport-introspection-cpp==3.1.4",
        "rosinstall==0.7.8",
        "rospkg==1.4.0",
        "rospkg-modules==1.5.0",
        "rpyutils==0.2.1",
        "rqt-action==2.0.1",
        "rqt-bag==1.1.4",
        "rqt-bag-plugins==1.1.4",
        "rqt-console==2.0.2",
        "rqt-graph==1.3.0",
        "rqt-gui==1.1.4",
        "rqt-gui-py==1.1.4",
        "rqt-msg==1.2.0",
        "rqt-plot==1.1.2",
        "rqt-publisher==1.5.0",
        "rqt-py-common==1.1.4",
        "rqt-py-console==1.0.2",
        "rqt-reconfigure==1.1.1",
        "rqt-service-caller==1.0.5",
        "rqt-shell==1.0.2",
        "rqt-srv==1.0.3",
        "rqt-topic==1.5.0",
        "scipy==1.8.0",
        "SecretStorage==3.3.1",
        "sensor-msgs==4.2.3",
        "sensor-msgs-py==4.2.3",
        "service-identity==18.1.0",
        "setuptools==59.6.0",
        "shape-msgs==4.2.3",
        "six==1.16.0",
        "slam-toolbox==2.6.4",
        "smclib==3.0.2",
        "snowballstemmer==2.2.0",
        "sos==4.4",
        "soupsieve==2.3.1",
        "speedtest-cli==2.1.3",
        "sros2==0.10.4",
        "ssh-import-id==5.11",
        "statistics-msgs==1.2.1",
        "std-msgs==4.2.3",
        "std-srvs==4.2.3",
        "stereo-msgs==4.2.3",
        "stevedore==5.0.0",
        "sympy==1.9",
        "systemd-python==234",
        "teleop-twist-keyboard==2.3.2",
        "tf2-geometry-msgs==0.25.2",
        "tf2-kdl==0.25.2",
        "tf2-msgs==0.25.2",
        "tf2-py==0.25.2",
        "tf2-ros-py==0.25.2",
        "tf2-tools==0.25.2",
        "theora-image-transport==2.5.0",
        "toml==0.10.2",
        "topic-monitor==0.20.3",
        "trajectory-msgs==4.2.3",
        "turtlesim==1.4.2",
        "Twisted==22.1.0",
        "ubuntu-advantage-tools==8001",
        "ufoLib2==0.13.1",
        "ufw==0.36.1",
        "unattended-upgrades==0.1",
        "unicodedata2==14.0.0",
        "unique-identifier-msgs==2.2.1",
        "urllib3==1.26.5",
        "vcstool==0.3.0",
        "vcstools==0.1.42",
        "virtualenv==20.20.0",
        "virtualenv-clone==0.5.7",
        "virtualenvwrapper==4.8.4",
        "vision-msgs==4.1.0",
        "visualization-msgs==4.2.3",
        "wadllib==1.3.6",
        "webencodings==0.5.1",
        "wheel==0.37.1",
        "wstool==0.1.17",
        "xacro==2.0.8",
        "zipp==1.0.0",
        "zope.interface==5.4.0"
    ],
    "usb": [
        "NoLib"
    ],
    "uname": [
        "Linux bubble 5.15.0-1026-raspi #28-Ubuntu SMP PREEMPT Fri Mar 10 14:28:52 UTC 2023 aarch64 aarch64"
    ]
}
  • Humble - Ubuntu 22.04.2 LTS
  • depthai-ros : Tried built from latest source and repository
  • depthai/depthai-core : Tried built from source and rosdep
Packages 
Package: ros-humble-depthai
Version: 2.20.2-1jammy.20230202.204256
Priority: optional
Section: misc
Maintainer: Sachin Guruswamy <[email protected]>
Installed-Size: 20.1 MB
Depends: libc6 (>= 2.34), libgcc-s1 (>= 4.5), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 12), libopencv-dev, libusb-1.0-0-dev, nlohmann-json3-dev, ros-humble-ros-workspace
Homepage: https://www.luxonis.com/
Download-Size: 10.3 MB
APT-Manual-Installed: no
APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages
Description: DepthAI core is a C++ library which comes with firmware and an API to interact with OAK Platform

Package: ros-humble-depthai-ros
Version: 2.6.4-1jammy.20230303.010530
Priority: optional
Section: misc
Maintainer: sachin <[email protected]>
Installed-Size: 42.0 kB
Depends: ros-humble-depthai, ros-humble-depthai-bridge, ros-humble-depthai-examples, ros-humble-depthai-ros-driver, ros-humble-depthai-ros-msgs, ros-humble-ros-workspace
Download-Size: 5,232 B
APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages
Description: The depthai-ros package

Package: ros-humble-depthai-bridge
Version: 2.6.4-1jammy.20230302.223904
Priority: optional
Section: misc
Maintainer: Sachin Guruswamy <[email protected]>
Installed-Size: 69.4 MB
Depends: libc6 (>= 2.32), libgcc-s1 (>= 3.3.1), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgcodecs4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 11), libboost-dev, libopencv-dev, ros-humble-camera-info-manager, ros-humble-cv-bridge, ros-humble-depthai, ros-humble-depthai-ros-msgs, ros-humble-image-transport, ros-humble-rclcpp, ros-humble-robot-state-publisher, ros-humble-ros-environment, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-stereo-msgs, ros-humble-vision-msgs, ros-humble-xacro, ros-humble-ros-workspace
Download-Size: 13.4 MB
APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages
Description: The depthai_bridge package

Package: ros-humble-depthai-ros-msgs
Version: 2.6.4-1jammy.20230302.210403
Priority: optional
Section: misc
Maintainer: Sachin Guruswamy <[email protected]>
Installed-Size: 1,310 kB
Depends: libc6 (>= 2.17), libgcc-s1 (>= 3.3.1), libpython3.10 (>= 3.10.0), libstdc++6 (>= 11), ros-humble-fastcdr, ros-humble-builtin-interfaces, ros-humble-geometry-msgs, ros-humble-rclcpp, ros-humble-rosidl-default-generators, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-vision-msgs, ros-humble-ros-workspace
Download-Size: 106 kB
APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages
Description: Package to keep interface independent of the driver

Package: ros-humble-depthai-ros-driver
Version: 2.6.4-1jammy.20230303.005120
Priority: optional
Section: misc
Maintainer: Adam Serafin <[email protected]>
Installed-Size: 844 kB
Depends: libc6 (>= 2.34), libconsole-bridge1.0 (>= 1.0.1+dfsg2), libgcc-s1 (>= 3.3.1), libopencv-core4.5d (>= 4.5.4+dfsg), libopencv-imgproc4.5d (>= 4.5.4+dfsg), libopencv-stitching4.5d (>= 4.5.4+dfsg), libstdc++6 (>= 11), ros-humble-ament-cmake-auto, ros-humble-camera-calibration, ros-humble-cv-bridge, ros-humble-depthai, ros-humble-depthai-bridge, ros-humble-diagnostic-msgs, ros-humble-image-pipeline, ros-humble-image-transport, ros-humble-image-transport-plugins, ros-humble-rclcpp, ros-humble-rclcpp-components, ros-humble-sensor-msgs, ros-humble-std-msgs, ros-humble-std-srvs, ros-humble-vision-msgs, ros-humble-ros-workspace
Download-Size: 213 kB
APT-Sources: http://packages.ros.org/ros2/ubuntu jammy/main arm64 Packages
Description: Depthai ROS Monolithic node.

everskies avatar Apr 18 '23 20:04 everskies

Update:

Turns out that the data is being published, but only once every 10 or so launches of camera.launch.py.

I'm currently checking if this is a hardware issue (either power related, or something else)

Edit: Does not appear to be the case unfortunately, I've tried some examples from depthai-python with the IR flood and project set to its maximum value, and ran a model at the same time, everything was stable.

everskies avatar Apr 18 '23 21:04 everskies

Did you check the issue? There's nothing problem with ROS2 foxy but problem with ROS noetic. It doesn't publish imu at all when I run camera.launch. But stereo_inertial.launch in the dai_examples publish imu.

HanDaSeul avatar May 02 '23 10:05 HanDaSeul

Hi @everskies, from your debug information it seems that your ROS packages are a little behind the official ones, could you upgrade and check again? Also, which camera model is that? @HanDaSeul same here, could you post your camera model and packages version?

Serafadam avatar May 04 '23 09:05 Serafadam

@Serafadam Replied here. https://github.com/luxonis/depthai-ros/issues/304#issuecomment-1533244754

HanDaSeul avatar May 04 '23 10:05 HanDaSeul