how to generate the depth image??

[jxr041100]

(1) we download the kitti gray sequences, because they are stereo, we firstly use the elas to convert stereo image into depth. one question, I saw the depth data you provided have the following property: close objects are dark, far objects are light. when I use the elas, how I can save the depth as the format you have?

[jxr041100]

it looks like the elas software only outputs the disparity map, not depth image, right? in order to generate the depth image, we should have extra effort.

[shichaoy]

@jxr041100 As mentioned in "preprocess_data/README.md", the depth image is CV_16UC1 format in millimeter unit. Closer object has small pixel value, thus more darker. The raw elas code only computes disparity, you need to change to depth by: depth = baseline*focal/disparity. Then threshold it by max depth (eg: 50m), x1000 to get mm unit, finally cast into CV_16UC1 image format.

[jxr041100]

@shichaoy In our experiment with kitti_00 sequences, we can not achieve the 3D map, I just wonder why it does not work. the first problem is depth generation from our code.

outpath = inpath[0:-4] + "_dep" + inpath[-4:] focal_len = float(args.focal_len) base_line = float(args.base_line) dis_img = cv.imread(inpath, cv.IMREAD_ANYCOLOR) rows, columns = dis_img.shape[0:2] ret, filter = cv.threshold(dis_img, 1, 255, cv.THRESH_BINARY) dis_filtered = filter / 255 *dis_img dephold = focal_len * base_line depth_out = dephold / dis_filtered depth_out = np.array(depth_out, dtype = "uint16") scale = 5 threshold = 65535 / scale depth_out[depth_out > threshold] = 0 depth_img = depth_out * scale depth_img = np.array(depth_img, dtype = "uint16")

cv.imwrite(outpath, depth_img)

when we get the depth = baseline*focal/disparity, how to threshold the depth? can you have the example code to show here? I saw in your code, you are using depth_scaling, how you define that value? can you help me look at my code or you share your code to generate the depth value.

[shichaoy]

@jxr041100 I used c++. In the raw elas code, I looped over each disparity, if depth is not in range of [0.2, 50] m, I set the depth value to be 0. (could also use threshold function) Then finally depth_float.convertTo(depth_int,CV_16UC1,1000.0,0); 1000 is a scaling number. It could also be 2000, or others (depending on the desired resolution, for example *1000 means the saved depth image resolution is 1mm, also pay attention not to overflow uint16 with max depth. I think my original image is thresholded by 30m, so I choose scaling 2000). The number should be the same when reading the image (in launch file). This is just my way of saving depth image, might be better ways.

I think in your code, after getting depth_out initially (I guess is actual float depth image), you should first threshold it using actual depth range such as [0.2,50], then times scaling, change to uint16, save to image.

[jxr041100]

@shichaoy thank you very much, very useful, and also thanks your sharing code.

[momonokochan]

@shichaoy thanks a lot for very interesting code. but I have some trouble when use elas_ros to create disparity. can you give me some guideline to use elas_ros?

[shichaoy]

@momonokochan As mentioned in the pre-process folder, it finally points to a ros package https://github.com/jeffdelmerico/cyphy-elas-ros. You can modify file elas.cpp to process online bag data or read images files. The interface is process() function.

[shichaoy]

this is part of main cpp I use. It should be enough.

On Wed, Feb 13, 2019 at 5:01 AM momonokochan [email protected] wrote:

@shichaoy https://github.com/shichaoy Thank you very much for the suggestion. I have try to use bag file get from kitti2bag to publish to elas_ros but I still cannot get disparity output in process() function. Can you give me your bag file or elas.cpp file you have modify?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/shichaoy/semantic_3d_mapping/issues/4#issuecomment-463190540, or mute the thread https://github.com/notifications/unsubscribe-auth/AGe8cRNPY7Dv8mcjdlI6_9DprvJVA7VCks5vNAyMgaJpZM4Wp7MZ .

[momonokochan]

I have create depth image follow your guideline but it has much noise . Can you give me some advice ? [

[yangsu9304]

I use the following code to check the depth info of your demo depth images cv2.imread('.../data_kitti/depth_img/000000.png',-1) and I found max = {uint16}65535 Is that right?
I think the max depth should be 50000 which we set before.

But my own depth_img results a worse 3D reconstruction.

[shichaoy]

@yangsu9304 I think my code use cv::imread(depth_img_name, CV_LOAD_IMAGE_ANYDEPTH), not sure it's same as -1. when I save image, I save as uint16, so each pixel max is 65535. I didn't see obvious problems in your image. It is usually sparse for just one frame, especially for far away points. if you have multiple frames, grid will be denser.

[yangsu9304]

@yangsu9304 I think my code use cv::imread(depth_img_name, CV_LOAD_IMAGE_ANYDEPTH), not sure it's same as -1. when I save image, I save as uint16, so each pixel max is 65535. I didn't see obvious problems in your image. It is usually sparse for just one frame, especially for far away points. if you have multiple frames, grid will be denser.

Thanks for your replying. As your previous answer, firstly threshold the depth image using actual depth range such as [0.2,50], then times scaling like 1000, change to uint16, save to image.
So the final depth max should be small than 50 * 1000 (or 30 * 2000). But why the depth max of your demo depth images from data_kitti is 65535? whcih means my own depth image is always darker than demo. my own depth image demo depth image

And in my 3D reconstruction the deformation of the yellow pole is too bad. The reconstruction is stretched along the road, when I threshold 30, scale 2000. In your launch file L19:

# pay attention to scaling, for cnn's image use 1000, for elas depth_img use 2000

[yangsu9304]

I have create depth image follow your guideline but it has much noise . Can you give me some advice ? [

maybe you didn't save as uint16

[shichaoy]

@yangsu9304 sorry for the confusions, I think I might mix different code versions. actually we could also don't do depth threshold, just save 65535 if depth is too large. or assign to 0. as long as we are aware of the depth limit. When we back-project points to 3D, too far away points won't be used, because it's out of range of the grid box.