ScanNet Depth images not aligning well

Hi, I'm trying to get the depth to align between images. However, my results typically look something like this:

When swapping between the images it is quite clear that they are not properly aligned. I'm using the provided depth, image, and intrinsics and pose files. Is there some post-processing step that is supposed to be done?

Feb 14 '22 20:02 Parskatt

https://github.com/ScanNet/ScanNet/issues/28 I guess the issue is that I first need to adjust by the extrinsics? Perhaps this could be made more clear?

Feb 15 '22 13:02 Parskatt

Although it seems like they have already been aligned in this way? I don't understand how they can be so wrong after postprocessing?

Feb 15 '22 15:02 Parskatt

Hi, did you figure this out? I am trying to resolve the alignment issue between RGB image and depth map as well

Mar 20 '24 16:03 justachetan

Nope, I think scannet++ is the way to go if you want accuracy.

Mar 20 '24 16:03 Parskatt

Hi, there is a colorToDepthExtrinsics in each {scene_id}.txt. Have you used that?

However, according to https://github.com/ScanNet/ScanNet/issues/92, they are already aligned and no extrinsic matrix should be applied. I am a bit confused for this problem. Which mode are you using?

Apr 15 '24 12:04 RunsenXu