depthai-ros
depthai-ros copied to clipboard
luxonis
Error to compile Multi_cam branchs (core,ros and ros examples)
I installed main brand without errors (but is not working on OAKD and RPI4) So I'm trying install the multicam branch to make work OAKD-PRO-W with ov9782 color central sensor. I installed the Depthai-core multi_cam_suport branch, the deptai-ros multi_cam brach and depthai_ros-examples multicam branch with this error. Obviously I have installed the cv_bridge from apt, but I put the source into the workspace to compile it all together and find it, but it doesnt work either.
ubuntu@ubuntu:~/depthai_ws$ catkin_make -j3
Base path: /home/ubuntu/depthai_ws
Source space: /home/ubuntu/depthai_ws/src
Build space: /home/ubuntu/depthai_ws/build
Devel space: /home/ubuntu/depthai_ws/devel
Install space: /home/ubuntu/depthai_ws/install
####
#### Running command: "cmake /home/ubuntu/depthai_ws/src -DCATKIN_DEVEL_PREFIX=/home/ubuntu/depthai_ws/devel -DCMAKE_INSTALL_PREFIX=/home/ubuntu/depthai_ws/install -G Unix Makefiles" in "/home/ubuntu/depthai_ws/build"
####
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /home/ubuntu/depthai_ws/devel
-- Using CMAKE_PREFIX_PATH: /home/ubuntu/livox_ws/devel;/opt/ros/noetic
-- This workspace overlays: /home/ubuntu/livox_ws/devel;/opt/ros/noetic
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.8.10", minimum required is "3")
-- Using PYTHON_EXECUTABLE: /usr/bin/python3
-- Using Debian Python package layout
-- Found PY_em: /usr/lib/python3/dist-packages/em.py
-- Using empy: /usr/lib/python3/dist-packages/em.py
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/ubuntu/depthai_ws/build/test_results
-- Forcing gtest/gmock from source, though one was otherwise available.
-- Found gtest sources under '/usr/src/googletest': gtests will be built
-- Found gmock sources under '/usr/src/googletest': gmock will be built
-- Found PythonInterp: /usr/bin/python3 (found version "3.8.10")
-- Found Threads: TRUE
-- Using Python nosetests: /usr/bin/nosetests3
-- catkin 0.8.10
-- BUILD_SHARED_LIBS is on
-- BUILD_SHARED_LIBS is on
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~ traversing 4 packages in topological order:
-- ~~ - cv_bridge
-- ~~ - depthai_ros_msgs
-- ~~ - depthai_bridge
-- ~~ - depthai_examples
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'cv_bridge'
-- ==> add_subdirectory(luxonis/cv_bridge)
-- Found PythonLibs: /usr/lib/aarch64-linux-gnu/libpython3.8.so (found version "3.8.10")
-- Found Boost: /usr/lib/aarch64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: python
-- Found OpenCV: /usr/local (found suitable version "4.5.5", minimum required is "4") found components: opencv_core opencv_imgproc opencv_imgcodecs
-- Found PythonLibs: /usr/lib/aarch64-linux-gnu/libpython3.8.so (found suitable version "3.8.10", minimum required is "3.8")
-- +++ processing catkin package: 'depthai_ros_msgs'
-- ==> add_subdirectory(luxonis/depthai-ros/depthai_ros_msgs)
-- Using these message generators: gencpp;geneus;genlisp;gennodejs;genpy
-- depthai_ros_msgs: 6 messages, 2 services
-- +++ processing catkin package: 'depthai_bridge'
-- ==> add_subdirectory(luxonis/depthai-ros/depthai_bridge)
-- Found Boost: /usr/lib/aarch64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0")
-- Could NOT find cv_bridge (missing: cv_bridge_DIR)
-- Could not find the required component 'cv_bridge'. The following CMake error indicates that you either need to install the package with the same name or change your environment so that it can be found.
CMake Error at /opt/ros/noetic/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
Could not find a package configuration file provided by "cv_bridge" with
any of the following names:
cv_bridgeConfig.cmake
cv_bridge-config.cmake
Add the installation prefix of "cv_bridge" to CMAKE_PREFIX_PATH or set
"cv_bridge_DIR" to a directory containing one of the above files. If
"cv_bridge" provides a separate development package or SDK, be sure it has
been installed.
Call Stack (most recent call first):
luxonis/depthai-ros/depthai_bridge/CMakeLists.txt:124 (find_package)
-- Configuring incomplete, errors occurred!
See also "/home/ubuntu/depthai_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/ubuntu/depthai_ws/build/CMakeFiles/CMakeError.log".
Invoking "cmake" failed
I just had installed main branch and it was working, and reinstalled cv_bridge from apt and I did include the source in the workspace too, just in case. everything failed with same error. I will try in other computer x86 tomorrow, main, and multi_cam, just like I did today with Rpi4
I checked my cv_bridge installation, upgraded via apt, and compiled darknet_ros and it compiled, so the cv_bridge is ok
Tried depthai-core muti_cam_suoport, with any combination of depthai- ros ans examples gives thw same error. Depthai-main, I guess it works as it was installed before
Yes I try it between the next two hours and let you know. If you want I try it in x86 let me know.
if you can confirm it. I will merge them to main
It worked now in 720p and 400p in OAK-D and looks fine, only that complaint about the IMU and the marker option when is on it looks do nothing, or I don't now what plug-in to use to visualize the marker. I'm using depthai-core main branch
It didn't work in OAK-D Pro W Dev, it says color sensor don't found. Any chance this work with depthai-core multi_cam_support instead main to have any chance to make work the ov9782? I don't need even stereo only the rgb because it is fixed focus,global shutter and fisheye and I need pair it with a 3D lidar to perform the VO and colorize lidar pointclouds.
My OAK-D is still failing randomly in 5.1V 3A power supplies, in the only ones that works consistent is with 5.2V and 4A. It does't work even with my Lenovo special charge USB port , I think you going to have more people with this problem, I hope that not, the camera gets quite hotter too compared with running other things, Maybe would be a good idea put a delay between the camera boot up and the NN process, I think al together make a spike of current that shutdown the usb3 communication.
if you can confirm it. I will merge them to main
Well this combination of branch if effectively fix the OAK-D issues, make two new issues. First, it breaks rtabmap, it looks the change in scale in rgb alignment, I guess done to fix the 400p issue, makes that the camera_info resolution is different to the one that arrives and rtabmap crash. Second, the black bands come back, homography is not being applied, which again bring us to a previous issue that invent false points. Both things are a step back, please take it in consideration before merge in main.
I think this is the line that make crash rtabmap, I guess camera_info is not taking that change, but really I don't know, is what I understand rtabmap is complaining
camRgb->setIspScale(1, 3);
camRgb->setIspScale(2, 5);
} else {
only that complaint about the IMU and the marker option when is on it looks do nothing
it is just a visualization for IMU. you can uncheck it to disable it.
It didn't work in OAK-D Pro W Dev, it says color sensor don't found.
On depthai-core use this branch multi_cam_manip_refactor and on depthai-ros-examples checkout ov9782 branch and use stereo_inertial_node. Currently only 720p works on that for RGB. I need to still work on 400p for RGB.
First, it breaks rtabmap, it looks the change in scale in rgb alignment, I guess done to fix the 400p issue, makes that the camera_info resolution is different to the one that arrives and rtabmap crash.
I will check on that.
Second, the black bands come back, homography is not being applied, which again bring us to a previous issue that invent false points.
Black band comes back because the RGB camera is 12MP and has different field of view. So to match its field of view depth map si slightly shrunk so there will be some empty space. We cannot do it such that yolo gets 416x416 resolution and crop the RGB sensor to match the same FOV to match the stereo at smaller resolution.
only that complaint about the IMU and the marker option when is on it looks do nothing
it is just a visualization for IMU. you can uncheck it to disable it.
It didn't work in OAK-D Pro W Dev, it says color sensor don't found.
On depthai-core use this branch
multi_cam_manip_refactorand on depthai-ros-examples checkoutov9782branch and use stereo_inertial_node. Currently only 720p works on that for RGB. I need to still work on 400p for RGB.
Ok , I will test only in 720p and tell you what I find in case is useful for you.
First, it breaks rtabmap, it looks the change in scale in rgb alignment, I guess done to fix the 400p issue, makes that the camera_info resolution is different to the one that arrives and rtabmap crash.
I will check on that.
Second, the black bands come back, homography is not being applied, which again bring us to a previous issue that invent false points.
Black bands came back because the RGB camera is 12MP and has different field of view. So to match its field of view depth map si slightly shrunk so there will be some empty space. We cannot do it such that yolo gets 416x416 resolution and crop the RGB sensor to match the same FOV to match the stereo at smaller resolution.
The black bands came back with rgb alignment turned off and I think I had NN desactivated too. When there is RGB alignment ON, I have not black bands and the depth is ok and the pointclouds too, it is probably because the 400p is cropped to 360p to align and cut the black bands off (just a theory), the problem is when it is working just like stereo, or you want colourize the pointclouds in grey to minimize CPU load(this is my case). If I use stereo_node , I think it was fine ,but I cannot use it as I lose some features as IMU.
I will test everything again (stereo nodes too to compare), and make pictures and videos if needed.
As you can see in this video with rgb alignment as it was in main branch the last two weeks worked ok, probably by such downgrade from 400 to 360
https://youtu.be/3LUTzJMUWTA
in normal mode. Black band at the corner is expected I think due to rectification unless you want to crop it off. I think in DIscord you mentioned you don't want to crop it off.
Hi, I trie the branches you said me for oak-d-pro. you didn't specify one for depthai-ros so I tested with 3 of them, multi_cam_supports fails with the next error:
`[ 77%] Building CXX object luxonis/depthai-ros-examples/depthai_examples/CMakeFiles/rgb_stereo_node.dir/ros1_src/rgb_stereo_node.cpp.o
In file included from /home/ubuntu/depthai_ws/src/luxonis/depthai-ros-examples/depthai_examples/ros1_src/rgb_stereo_node.cpp:14:
/home/ubuntu/depthai_ws/src/luxonis/depthai-ros/depthai_bridge/include/depthai_bridge/BridgePublisher.hpp: In instantiation of ‘void dai::ros::BridgePublisher<RosMsg, SimMsg>::publishMetadata(std::shared_ptr<Tp>, const RosMsg&) [with RosMsg = sensor_msgs::Image<std::allocator
/home/ubuntu/depthai_ws/src/luxonis/depthai-ros/depthai_bridge/include/depthai_bridge/BridgePublisher.hpp:369:38: error: ‘using element_type = class dai::ImgFrame’ {aka ‘class dai::ImgFrame’} has no member named ‘getExposureTime’
369 | msg.exposure_time = inFrame->getExposureTime();
| ~~~~~~~~~^~~~~~~~~~~~~~~
/home/ubuntu/depthai_ws/src/luxonis/depthai-ros/depthai_bridge/include/depthai_bridge/BridgePublisher.hpp:371:36: error: ‘using element_type = class dai::ImgFrame’ {aka ‘class dai::ImgFrame’} has no member named ‘getSensitivity’
371 | msg.sensitivity = inFrame->getSensitivity();
| ~~~~~~~~~^~~~~~~~~~~~~~
/home/ubuntu/depthai_ws/src/luxonis/depthai-ros/depthai_bridge/include/depthai_bridge/BridgePublisher.hpp: In instantiation of ‘void dai::ros::BridgePublisher<RosMsg, SimMsg>::publishMetadata(std::shared_ptr<Tp>, const RosMsg&) [with RosMsg = stereo_msgs::DisparityImage<std::allocator
/home/ubuntu/depthai_ws/src/luxonis/depthai-ros/depthai_bridge/include/depthai_bridge/BridgePublisher.hpp:369:38: error: ‘using element_type = class dai::ImgFrame’ {aka ‘class dai::ImgFrame’} has no member named ‘getExposureTime’
369 | msg.exposure_time = inFrame->getExposureTime();
| ~~~~~~~~~^~~~~~~~~~~~~~~
/home/ubuntu/depthai_ws/src/luxonis/depthai-ros/depthai_bridge/include/depthai_bridge/BridgePublisher.hpp:371:36: error: ‘using element_type = class dai::ImgFrame’ {aka ‘class dai::ImgFrame’} has no member named ‘getSensitivity’
371 | msg.sensitivity = inFrame->getSensitivity();
| ~~~~~~~~~^~~~~~~~~~~~~~
In file included from /usr/include/c++/9/numeric:62,
from /usr/local/lib/cmake/depthai/dependencies/include/nop/base/map.h:21,
from /usr/local/lib/cmake/depthai/dependencies/include/nop/serializer.h:24,
from /usr/local/include/depthai-shared/utility/Serialization.hpp:9,
from /usr/local/include/depthai-shared/common/Point3f.hpp:7,
from /usr/local/include/depthai-shared/common/Extrinsics.hpp:6,
from /usr/local/include/depthai-shared/common/CameraInfo.hpp:4,
from /usr/local/include/depthai-shared/common/EepromData.hpp:7,
from /usr/local/include/depthai/device/CalibrationHandler.hpp:6,
from /usr/local/include/depthai/depthai.hpp:9,
from /home/ubuntu/depthai_ws/src/luxonis/depthai-ros-examples/depthai_examples/ros1_src/rgb_stereo_node.cpp:12:
/usr/include/c++/9/bits/stl_numeric.h:158:5: error: ‘_Tp std::accumulate(_InputIterator, _InputIterator, _Tp, _BinaryOperation) [with _InputIterator = __gnu_cxx::__normal_iterator<const float*, std::vector
`
The other two I tested are main and fix-find-package branches, they both failed with a different error
I'm not busy so if is useful for you any other test, just let me know I will happy to test