Mismatch between comment and code

[christian-lanius]

Hi, Thank you for sharing your code with us! I am trying to evaluate the method on our own dataset. We gathered larger images and thus have to crop/resize them. When looking at the code, the comment in kitti_loader.py states:

note: we will take the center crop of the images during augmentation # that changes the optical centers, but not focal lengths https://github.com/fangchangma/self-supervised-depth-completion/blob/master/dataloaders/kitti_loader.py#L29

The optical center is then adjusted. However, in lines 145 and 168, a bottom crop is applied to the images. Thus, if I understand the code correctly, the full crop distance has to be subtracted from the focal centers.

Can you check if my understanding in this regard is correct?

Kind regards, Chris

[fangchangma]

Hey Chris,

Thanks for point this out -- the comment was outdated, and we are indeed doing bottom crop (same as the KITTI dataset).

[christian-lanius]

So does this mean that the adaptation of the intrinsics should be changed to:

K[0,2] = K[0,2] - 26 # from width = 1242 to 1216, with a 13-pixel cut on both sides
K[1,2] = K[1,2] - 23 # from width = 375 to 352, with a 11.5-pixel cut on both sides

I crop my image to owidth, oheight, and then scale it by a factor imScale. Thus I changed the code in the kitti_loader as such (with my own intrinsics):

    orig_x, orig_y = 2208, 1242
    K = np.zeros((3,3))
    fx, fy, cx, cy = 1399.87, 1399.87, 1056.62, 597.53
    K[0,0] = fx
    K[0,2] = cx - (orig_x - owidth)/2
    K[1,1] = fy
    K[1,2] = cy - (orig_y - oheight)/2
    K = scale*K
    
    K[2,2] = 1
    return K

and in main.py@L82 to:

kitti_intrinsics = Intrinsics(int(owidth*imScale), int(oheight*imScale), fu, fv, cu, cv).cuda()

Are there any other locations in the code where the intrinsics are used? I did not find any, but I am experiencing similar problems to #19

[fangchangma]

Are there any other locations in the code where the intrinsics are used?

There is an Intrinsics class in inverse_warp.py that performs similarly to your code. https://github.com/fangchangma/self-supervised-depth-completion/blob/898aa91f569adb1165fee82d4318025fae78546a/inverse_warp.py#L5

If you use VLP-32 as input, it is not surprising that the pretrained model does not work (since it was trained on HDL-64 lidars). Some finetuning on your own dataset might be necessary.

[windyrobin]

 # note: we will take the center crop of the images during augmentation
    # that changes the optical centers, but not focal lengths
    K[0, 2] = K[
        0,
        2] - 13  # from width = 1242 to 1216, with a 13-pixel cut on both sides
    K[1, 2] = K[
        1,
        2] - 11.5  # from width = 375 to 352, with a 11.5-pixel cut on both sides

so the code should be changed ?

[longyangqi]

When doing BottomCrop， I think it should be:

K[0, 2] = K[0, 2] - 13
K[1, 2] = K[1, 2] - 23

I don't know if it is correct. Thanks！