depthai-ros
depthai-ros copied to clipboard
luxonis
camera_info P matrices are zero
I launched depthai_examples rgb_stereo_node.launch and I noticed the camera_info of both the RGB and stereo cameras have all zeros in the P component. This practically makes depth_image_proc unusable with the images.
Example:
header:
seq: 6333
stamp:
secs: 1642079111
nsecs: 580556369
frame_id: "oak_rgb_camera_optical_frame"
height: 720
width: 1280
distortion_model: "rational_polynomial"
D: [1.2615910768508911, -57.75300216674805, 0.0008713369606994092, 0.0013280882267281413, 282.6454772949219, 1.1072325706481934, -56.35382843017578, 277.4756164550781]
K: [1033.0888671875, 0.0, 646.242919921875, 0.0, 1031.7818603515625, 369.31561279296875, 0.0, 0.0, 1.0]
R: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
P: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
binning_x: 0
binning_y: 0
roi:
x_offset: 0
y_offset: 0
height: 0
width: 0
do_rectify: False
# This holds for both images of a stereo pair.
float64[12] P # 3x4 row-major matrix
From camera_info msg.
RGB camera is not part of the stereo pair hence it is skipped in the RGB. We are just aligning the depth in right to the RGB.
Can you provide the details on how this affects the depth_image_proc ?
Well, The P matrix isn't filled for the stereo camera either.
Given the fact that both K and R matrices are filled for the cameras, computing the P matrix is just a matter of multiplying them.
In depth_image_proc, I don't get any output from point_cloud_xyz or point_cloud_xyzrgb because the pinhole camera model it uses depends on P: https://github.com/ros-perception/vision_opencv/blob/c791220cefd0abf02c6719e2ce0fea465857a88e/image_geometry/src/pinhole_camera_model.cpp#L315 . Also, try to insert a "Camera" type view in rviz - you will also get nothing and the view will warn that the P matrix is all-zero.
we do publish p when it is in stereo camera. In this example it is not there because it is aligned with RGB. But I get what you mean.
Will add the P for that too. Sorry for the issue.