Camera trace failed on custom dataset

[Contrail97]

First of all, thank you for your research, which has been very inspiring to me. I found the reconstruction results obtained using your project on the replica dataset satisfactory. However, I encountered some issues when working with a custom dataset. Due to certain reasons, I couldn't use the online or offline modes of NerfCapture. Therefore, I used Polycam instead of NerfCapture to capture scenes and generate depth, RGB, and transforms.json.

Then I modified the following config in file configs/iphone/splatam.py： base_dir = "./data" scene_name = "poly1" num_frames = 188

And use the following instructions to reconstruct the scene python scripts/splatam.py configs/iphone/splatam.py

When I set the use_gt_poses = True, I found that the reconstructed room barely met its original geometric constraints:

But when I set the use_gt_poses = False, I noticed that the reconstructed scenes were starting to look strange: It looks like a spiral that strings together each trained rgb image on the line,i.e. the camera's pose estimate has shifted a lot.

At first, I thought there was something wrong with the depth map, so I opened the eval directory to see the evaluation screenshot:


From the screenshot above it looks like there's nothing wrong with the depth map. So I don't know, where did I go wrong? If there is any suggestion I will be very appreciated.

I also upload my custom dataset on Google to reproduce the issue. https://drive.google.com/file/d/1ogvhCim9vXZjVK0EpqAqTat93bAbUblp/view?usp=sharing

Thanks, look forward to your reply.

[Zhangyangrui916]

When i set the use_gt_poses = True, I fix wrong camera pose in my custom dataset, which takes me several days to solve…… I wonder why use_gt_poses default value is False ?

You can try my dataset to see if it work https://pan.baidu.com/s/1gsUwh0x9pH2cy78G97JjUg?pwd=gljt

i captured this data with a modified version of NerfCapture, you can check my repo

[Nik-V9]

Hi, Thanks for trying the code out with Polycam data!

I think the tracking is failing because the images are sideways. I would recommend orienting them correctly and trying.