Questions regarding the cam param file and roslaunch

[FaboNo]

Thank you very much for sharing this visual odometry software

I would like to use a D435i in a ROS environment. I carefully read the documentation and I successfully compiled the ROS Kinetic version.

Next step is the configuration file, I saw that there is a template (just include the information related to the camera data):

#==============#
Camera Model
#==============#

Camera.name: "Realsense 435i"
Camera.setup - can change into stereo or RGBD

Camera.setup: "RGBD"
Camera.model: "perspective"
parameters for RGB image (change according to your camera calibration)

Camera.fx: 608.958435
Camera.fy: 608.723389
Camera.cx: 321.793488
Camera.cy: 240.326843

Camera.k1: 0.0
Camera.k2: 0.0
Camera.p1: 0.0
Camera.p2: 0.0
Camera.k3: 0.0
parameters for stereo setup. (change according to your camera calibration)

LEFT.height: 480
LEFT.width: 640
LEFT.D: !!opencv-matrix
rows: 1
cols: 5
dt: d
data: [-0.015627, 0.000299, 0.000140, 0.000878, 0.000000]
LEFT.K: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 1.0]
LEFT.R: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
LEFT.P: !!opencv-matrix
rows: 3
cols: 4
dt: d
data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 0.0, 1.0, 0.0]

RIGHT.height: 480
RIGHT.width: 640
RIGHT.D: !!opencv-matrix
rows: 1
cols: 5
dt: d
data: [0.0, 0.0, 0.0, 0.0, 0.0]
RIGHT.K: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 1.0]
RIGHT.R: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
RIGHT.P: !!opencv-matrix
rows: 3
cols: 4
dt: d
data: [382.80267333984375, 0.0, 316.8811340332031, 0.0, 0.0, 382.80267333984375, 237.82000732421875, 0.0, 0.0, 0.0, 1.0, 0.0]

Camera.fps: 30.0
Camera.cols: 640
Camera.rows: 480
Camera.focal_x_baseline: 18.39835242 # please change this parameter your_camera_fx*baseline

Here my questions are:

• Concerning Camera.fx, Camera.fy,Camera.cx,Camera.cy fields, do we have to put a value here or not? As I deal with a stereo camera, I have to the left and right camera, each one with its specific intrinsics and distortion parameters?
• Camera model = "perspective" means pinhole or not?
• how to compute Camera.focal_x_baseline? As we have two different camera_fx, which one do we take (baseline is 0.05 for the D435i).

Once the .yaml file has been filed in, we have to launch rosrun openvslam run_slam -v /path/to/orb_vocab.dbow2 -c /path/to/config.yaml but has we are working with a camera, there is no need to rosrun a publisher but instead remap /camera/left to /camera/infra1 do you confirm? And probably create a launch file later on.

Finally where can I find the position and the "point clouds" in order to include them in the ros wrapper - so that I can publish them as topics?

Thank you again for this tremendous work

[mirellameelo]

@FaboNo Hi!

Once you have the projection matrix (P1 and P2) of your cameras (You can find it for example by using the opencv function StereoRectify or ROS camera calibration package), then:

camera.fx = P1[0][0] or P2[0][0] camera.fy = P1[1][1] or P2[1][1] camera.cx = P1[0][2] or P2[0][2] camera.cy = P1[1][2] or P2[1][2]

Camera.focal_x_baseline = camera.fx * baseline

EDIT: I'm considering this template.

Once you run run_slam, the node will wait for images. I'm not sure which version you using, so you should check 1) if the package you are using supports stereo camera and 2) the subscribers' topic names. For example: Here you can find a version that supports ROS and subscribes to the images on camera/left/image_raw and camera/right/image_raw. So, make sure you are publishing your images on these topics.

If you want to publish the point cloud, I think this issue #448 can help you.

[FaboNo]

@mirellameelo thank you for your message. Actually I built a launch file and I am able to run the stereo as it is shown below:

But looks like Pangolin is offering only a 2D view? I am not familiar with Pangolin so I do not know it is possible to turn in a 3D view?

Regarding the ROS odom topic, I followed your link and I saw the example for the monoCam example and I was able to reproduce with Ros1 for the stereo class (but I think there is a problem with the axis because when I carry up the camera I can see the Y axis value changing but nothing on the Z axis).

However regarding the example to display the PointCloud I see that I need to add a method in the system class, so I will test it and let you know

Thank you again for your help

[mirellameelo]

@FaboNo Pangolin has 3D visualization. Left-button translates the scene, and the middle-button rotates it. I don't know how to do it without a mouse... =/

Once you carry up the camera, is the axis Y which should be moving, so you're good! The plane is XZ, not XY as we are used for. rs

You're welcome!

[FaboNo]

@mirellameelo I looked at the way the odometry has been computed and I have few questions:

• The homogeneous matrix which is returned by auto cam = SLAM_.feed_stereo_frame(leftcv, rightcv, timestamp, mask_); is associated with the left or right camera?
• Is there any document describing the frame orientation, although I can deduce it by looking at the Rotation part ?
• Actually ROS ENU et x axis Front and Z Up so I move the camera up and down I should see the z value changing over time
• Regarding the orientation, I think that the RPY to quaternion operation is missing I will continue my investigation and let you know Regards

[mirellameelo]

@FaboNo hi! For now, I'm using a monocular camera, but in your case, I believe it regards the left camera - I'm not sure tho. Regarding the orientation, I plot the odometry in RVIZ and seems pretty consistent, and its calculus considers the quaternion. Not sure if I understood it right, sorry. Let me know the news about it. Regards.