openvslam
openvslam copied to clipboard
xdspacelab
RealSense D435i config.yaml for run_slam ROS node
Hi,
Thanks for sharing this great project!
I'm trying to the OpenVslam run_slam node with my RealSense D435i camera on Ubuntu 18.04 but I don't know how I can write the config.yaml file.
Any help?
@mhaboali Realsense D435i has depth and stereo. Which type you want to try ? You just need to input your calibrated realsense parameters, if you cannot calibrate it, you can try echo the topics of camera_info publish by realsense ros package. If you run stereo setup, you need to put the baseline of the camera which is 0.49. Please try editing ros run_slam node to support the stereo and rgdb. you can refer some of examples that given in the repo (not ros).
Hi,
Thanks for sharing this great project!
I'm trying to the OpenVslam run_slam node with my RealSense D435i camera on Ubuntu 18.04 but I don't know how I can write the config.yaml file.
Any help?
Have you managed to create config file like this? Can you share it?
@shirbarzel @mhaboali
#==============#
Camera Model
#==============#
Camera.name: "Realsense 435i"
Camera.setup - can change into stereo or RGBD
Camera.setup: "RGBD" Camera.model: "perspective"
parameters for RGB image (change according to your camera calibration)
Camera.fx: 608.958435 Camera.fy: 608.723389 Camera.cx: 321.793488 Camera.cy: 240.326843
Camera.k1: 0.0 Camera.k2: 0.0 Camera.p1: 0.0 Camera.p2: 0.0 Camera.k3: 0.0
parameters for stereo setup. (change according to your camera calibration)
LEFT.height: 480 LEFT.width: 640 LEFT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [-0.015627, 0.000299, 0.000140, 0.000878, 0.000000] LEFT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 1.0] LEFT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] LEFT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 0.0, 1.0, 0.0]
RIGHT.height: 480 RIGHT.width: 640 RIGHT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [0.0, 0.0, 0.0, 0.0, 0.0] RIGHT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 1.0] RIGHT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] RIGHT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 0.0, 1.0, 0.0]
Camera.fps: 30.0 Camera.cols: 640 Camera.rows: 480 Camera.focal_x_baseline: 18.39835242 # please change this parameter your_camera_fx*baseline
Camera.color_order: "RGB"
#================#
ORB Parameters
#================#
Feature.max_num_keypoints: 1000 Feature.scale_factor: 1.2 Feature.num_levels: 8 Feature.ini_fast_threshold: 20 Feature.min_fast_threshold: 7
#=====================#
Tracking Parameters
#=====================#
depth_threshold: 10.0 depthmap_factor: 1000.0
#========================#
Initializer Parameters
#========================#
Initializer.num_min_triangulated_pts: 400
#===========================#
PangolinViewer Parameters
#===========================#
PangolinViewer.keyframe_size: 0.07 PangolinViewer.keyframe_line_width: 1 PangolinViewer.graph_line_width: 1 PangolinViewer.point_size: 2 PangolinViewer.camera_size: 0.08 PangolinViewer.camera_line_width: 3 PangolinViewer.viewpoint_x: 0 PangolinViewer.viewpoint_y: -0.65 PangolinViewer.viewpoint_z: -1.9 PangolinViewer.viewpoint_f: 400
#=====================#
Tracking Parameters
#=====================#
#depth_threshold: 40
Thanks so much @Shashika007 it's really appreciated to get such important information for all users.
Once I get a chance to test it and I'll share my experience with it.
@shirbarzel @mhaboali
#==============#
Camera Model
#==============#
Camera.name: "Realsense 435i"
Camera.setup - can change into stereo or RGBD
Camera.setup: "RGBD" Camera.model: "perspective"
parameters for RGB image (change according to your camera calibration)
Camera.fx: 608.958435 Camera.fy: 608.723389 Camera.cx: 321.793488 Camera.cy: 240.326843
Camera.k1: 0.0 Camera.k2: 0.0 Camera.p1: 0.0 Camera.p2: 0.0 Camera.k3: 0.0
parameters for stereo setup. (change according to your camera calibration)
LEFT.height: 480 LEFT.width: 640 LEFT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [-0.015627, 0.000299, 0.000140, 0.000878, 0.000000] LEFT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 1.0] LEFT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] LEFT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 0.0, 1.0, 0.0]
RIGHT.height: 480 RIGHT.width: 640 RIGHT.D: !!opencv-matrix rows: 1 cols: 5 dt: d data: [0.0, 0.0, 0.0, 0.0, 0.0] RIGHT.K: !!opencv-matrix rows: 3 cols: 3 dt: d data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 1.0] RIGHT.R: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0] RIGHT.P: !!opencv-matrix rows: 3 cols: 4 dt: d data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 0.0, 1.0, 0.0]
Camera.fps: 30.0 Camera.cols: 640 Camera.rows: 480 Camera.focal_x_baseline: 18.39835242 # please change this parameter your_camera_fx*baseline
Camera.color_order: "RGB"
#================#
ORB Parameters
#================#
Feature.max_num_keypoints: 1000 Feature.scale_factor: 1.2 Feature.num_levels: 8 Feature.ini_fast_threshold: 20 Feature.min_fast_threshold: 7
#=====================#
Tracking Parameters
#=====================#
depth_threshold: 10.0 depthmap_factor: 1000.0
#========================#
Initializer Parameters
#========================#
Initializer.num_min_triangulated_pts: 400
#===========================#
PangolinViewer Parameters
#===========================#
PangolinViewer.keyframe_size: 0.07 PangolinViewer.keyframe_line_width: 1 PangolinViewer.graph_line_width: 1 PangolinViewer.point_size: 2 PangolinViewer.camera_size: 0.08 PangolinViewer.camera_line_width: 3 PangolinViewer.viewpoint_x: 0 PangolinViewer.viewpoint_y: -0.65 PangolinViewer.viewpoint_z: -1.9 PangolinViewer.viewpoint_f: 400
#=====================#
Tracking Parameters
#=====================#
#depth_threshold: 40
Thank you so much! Can you also share which node are you running this file with? is it based on the "run_tum_rgbd_slam"? If you already did it can you share the exact instructions in order to run it with the D435i?
Can you share how you obtained the information? I have a Realsense R200 and I do not know how to create this config file, or where to get the information from you've mentioned.
@shirbarzel @jagauthier Inside folder /openvslam/example/tum_vi you can find a config file (.yalm extension) for both monocular camera and stereo. And inside /openvslam/example/tum_rgbd the RGBD config file. They can be useful as an example!
You can copy these and just change the parameters of the "camera model", for the parameters of your camera. To know they, take a look at this tutorial or this video tutorial, both using OpenCV, however, there are other ways to do it.
I was using the config file without change the camera parameters to the first tests, and it works. So in the last case... but it is not the best way out!!! hehe
If you are using ROS, you just need to indicate the path of your config file. Such as:
# ROS
rosrun openvslam run_slam -v /path/to/orb_vocab.dbow2 -c /path/to/config.yaml
# ROS 2
ros2 run openvslam run_slam -v /path/to/orb_vocab.dbow2 -c /path/to/config.yaml
Check out this #247 for more information on how to run in slam mode using your USB camera (ROS2). For ROS1 the documentation cover it!
Thanks! That makes sense, and I will have to build something for calibration, I read this here
It seems that I can use the OpenVSLAM examples okay, but when I tried to edit the camera things did not work. Then I tried one of the cameras configs you mentioned (TUM_RGBD_rgbd_1.yaml)
and that ends up crashing on me.
~$ rosrun openvslam run_slam -v /home/uav122/orb_vocab/orb_vocab.dbow2 -c /home/uav122/TUM_RGBD_rgbd_1.yaml
[2020-05-22 18:35:39.235] [I] config file loaded: /home/uav122/TUM_RGBD_rgbd_1.yaml
terminate called after throwing an instance of 'std::runtime_error'
what(): Invalid setup type: RGBD
*** Aborted at 1590172539 (unix time) try "date -d @1590172539" if you are using GNU date ***
PC: @ 0x7f71ff178428 gsignal
*** SIGABRT (@0x3e8000050b2) received by PID 20658 (TID 0x7f7202dc5900) from PID 20658; stack trace: ***
@ 0x7f7200c84390 (unknown)
@ 0x7f71ff178428 gsignal
@ 0x7f71ff17a02a abort
@ 0x7f71ff7b284d __gnu_cxx::__verbose_terminate_handler()
@ 0x7f71ff7b06b6 (unknown)
@ 0x7f71ff7b0701 std::terminate()
@ 0x7f71ff7b0919 __cxa_throw
@ 0x416785 main
@ 0x7f71ff163830 __libc_start_main
@ 0x4174d9 _start
@ 0x0 (unknown)
Aborted (core dumped)
@jagauthier hmmm, I used the monocular example... =/ Did you solve the camera calibration for your RGBD camera? I think you now depend on that to move forward in your project.
I have not calibrated yet. I'm just trying to make something work before putting forth the calibration effort. I can get a monocular camera to load.
I don't think it's strictly a calibration issue because of this:
Invalid setup type: RGBD
The code is rejecting that type. It's also rejecting stereo (I do have two IR cameras). I haven't dug into the code to see what it think the valid "setup" types are yet.
only monocular works for SLAM with ROS?
if (cfg->camera_->setup_type_ == openvslam::camera::setup_type_t::Monocular) {
mono_tracking(cfg, vocab_file_path->value(), mask_img_path->value(), eval_log->is_set(), map_db_path->value());
}
else {
throw std::runtime_error("Invalid setup type: " + cfg->camera_->get_setup_type_string());
}
Did you try to change it for your camera type? It can be "Monocular", "RGBD" and "Stereo". Don't forget you have to build the packages again.
Sadly, that's not going to work. There is no supporting code for rgbd_tracking() in the ROS portion of the code, and trying to add it did not work, as the image is obtained through a ROS subscription on /camera/image_raw. I also need to access the depth image of /camera/depth and that's not something I really know how to do, as I've very little ROS programming experience at this level.
Sadly, that's not going to work. There is no supporting code for
rgbd_tracking()in the ROS portion of the code, and trying to add it did not work, as the image is obtained through a ROS subscription on /camera/image_raw. I also need to access the depth image of /camera/depth and that's not something I really know how to do, as I've very little ROS programming experience at this level.
This is exactly the problem I also encountered, there seems to be no full interface for the D435i, no support for IMU as well.
@jagauthier =/ If you decide to move forward with this feature, it would be nice a share of your results and also difficulties. I'm still learning ROS, but I hope to be useful as I can.
@shinsumicco I believe this is a great point to consider for Openvslam project.
https://github.com/xdspacelab/openvslam/pull/276 adds ROS support for stereo and RGBD.
#276 adds ROS support for stereo and RGBD.
@ymd-stella Thanks, can you please explain how to run the relevant nodes with the D435i? Is there any support for imu data?
@ymd-stella How do I pull this in? Every time I try it creates new directories under the ros/ directory named '1','2', and the src folder it emptied out.
How do I pull this in? Every time I try it creates new directories under the ros/ directory named '1','2', and the src folder it emptied out.
After https://github.com/xdspacelab/openvslam/commit/7eb918c3a083ead36030b109369a5a975cfe71e1 , in the develop branch, the catkin workspace has changed from ros to ros/1.
Thanks, can you please explain how to run the relevant nodes with the D435i?
(I'm not sure what you mean by relevant nodes, but) the procedure is the same as https://github.com/xdspacelab/openvslam/issues/308#issuecomment-632825904 .
(#276 updates ROS1 wrapper only).
Is there any support for imu data?
No.
@ymd-stella Ah, for ROS1 and ROS2.
Unfortunately, it doesn't build.
[ 63%] Building CXX object openvslam/src/CMakeFiles/run_localization.dir/openvslam_ros.cc.o
/openvslam/ros/1/src/openvslam/src/openvslam_ros.cc: In constructor 'openvslam_ros::stereo::stereo(const std::shared_ptr<openvslam::config>&, const string&, const string&, bool)':
/openvslam/ros/1/src/openvslam/src/openvslam_ros.cc:36:42: error: missing template arguments before '(' token
sync_(message_filters::Synchronizer(SyncPolicy(10), left_sf_, right_sf_)) {
^
Thanks, can you please explain how to run the relevant nodes with the D435i?
(I'm not sure what you mean by
relevant nodes, but) the procedure is the same as #308 (comment) . (#276 updates ROS1 wrapper only).Is there any support for imu data?
No.
By relevant nodes I meant if yo turn the "run_slam" ros node like this:
rosrun openvslam run_slam -v /path/to/orb_vocab.dbow2 -c /path/to/config.yaml
- With orb_vocab.dbow2?
- With this config file : https://github.com/xdspacelab/openvslam/issues/308#issuecomment-628406238 ?
- Which topics of the D435i this node subscribes to ?
Thanks
With orb_vocab.dbow2?
Yes.
With this config file : #308 (comment) ?
Yes.
Which topics of the D435i this node subscribes to ?
https://github.com/xdspacelab/openvslam/blob/e0e2aac94249119574932f4fd849df9cc7f8ce30/ros/1/src/openvslam/src/openvslam_ros.cc#L34-L35
@jagauthier
Unfortunately, it doesn't build.
I can't reproduce it by the command below on ymd-stella:feat/ros-stereo.
docker build -t openvslam-ros -f Dockerfile.ros .
touch /tmp/.docker.xauth
xauth nlist $DISPLAY | sed -e 's/^..../ffff/' | xauth -f /tmp/.docker.xauth nmerge -
docker run -it --name openvslam-ros -e DISPLAY=$DISPLAY -e XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR} -e XAUTHORITY=/tmp/.docker.xauth -v /tmp/.X11-unix/:/tmp/.X11-unix:ro -v ${XDG_RUNTIME_DIR}:${XDG_RUNTIME_DIR} -v $HOME:/host:rw --net=host openvslam-ros
# in docker container
cd /openvslam/ros/1
catkin_make -j4 -DBUILD_WITH_MARCH_NATIVE=ON -DUSE_PANGOLIN_VIEWER=ON -DUSE_SOCKET_PUBLISHER=OFF -DUSE_STACK_TRACE_LOGGER=ON -DBOW_FRAMEWORK=DBoW2
Could you provide more information?
Yes, definitely.
I'm using ROS Kinetic on Ubuntu Xenial. Most of the ROS packages have been installed using the package manager.
The error looks like templating:
openvslam/ros/1/src/openvslam/src/openvslam_ros.cc: In constructor 'openvslam_ros::stereo::stereo(const std::shared_ptr<openvslam::config>&, const string&, const string&, bool)':
openvslam/ros/1/src/openvslam/src/openvslam_ros.cc:36:42: error: missing template arguments before '(' token
sync_(message_filters::Synchronizer(SyncPolicy(10), left_sf_, right_sf_)) {
So, I checked and I'm using gcc 5.4. It's a little dated, so maybe it's a compiler issue.
Edit: Upgraded to gcc-7.5 and have a different compilation error.
openvslam/ros/1/src/openvslam/src/openvslam_ros.cc: In constructor 'openvslam_ros::stereo::stereo(const std::shared_ptr<openvslam::config>&, const string&, const string&, bool)':
openvslam/ros/1/src/openvslam/src/openvslam_ros.cc:37:77: error: call of overloaded 'bind<void>(void (openvslam_ros::stereo::*)(const ImageConstPtr&, const ImageConstPtr&), openvslam_ros::stereo*, boost::arg<1>&, boost::arg<2>&)' is ambiguous
sync_.registerCallback(boost::bind<void>(&stereo::callback, this, _1, _2));
^
In file included from /usr/include/boost/bind.hpp:22:0,
from /opt/ros/kinetic/include/ros/publisher.h:35,
from /opt/ros/kinetic/include/ros/node_handle.h:32,
from /opt/ros/kinetic/include/ros/ros.h:45,
from openvslam/ros/1/src/openvslam/src/openvslam_ros.h:8,
from openvslam/ros/1/src/openvslam/src/openvslam_ros.cc:1:
/usr/include/boost/bind/bind.hpp:1610:5: note: candidate: boost::_bi::bind_t<R, F, typename boost::_bi::list_av_3<A1, A2, A3>::type> boost::bind(F, A1, A2, A3) [with R = void; F = void (openvslam_ros::stereo::*)(const boost::shared_ptr<const sensor_msgs::Image_<std::allocator<void> > >&, const boost::shared_ptr<const sensor_msgs::Image_<std::allocator<void> > >&); A1 = openvslam_ros::stereo*; A2 = boost::arg<1>; A3 = boost::arg<2>; typename boost::_bi::list_av_3<A1, A2, A3>::type = boost::_bi::list3<boost::_bi::value<openvslam_ros::stereo*>, boost::arg<1>, boost::arg<2> >]
BOOST_BIND(F f, A1 a1, A2 a2, A3 a3)
And on and on..I wonder if this is a boost issue? Looks like boost 1.58
@jagauthier thanks. I was able to reproduce it. Now that I've fixed it by 5ad0ab7.
But> > With orb_vocab.dbow2?
Yes.
With this config file : #308 (comment) ?
Yes.
Which topics of the D435i this node subscribes to ?
https://github.com/xdspacelab/openvslam/blob/e0e2aac94249119574932f4fd849df9cc7f8ce30/ros/1/src/openvslam/src/openvslam_ros.cc#L34-L35
@ymd-stella Hi.
I am using the ros project #276 to run "run_slam" node with Realsense D435. The ros project is rebuilt success.But after running the "run_slam" node, both fram viewer and map viewer do not have any reaction and keep blank.
I am using orb_vocab.dbow2 and config file #308.
Do you have any idea about such problem. Or does anybody have any idea? Many thanks.
Peace!!
