IGEV icon indicating copy to clipboard operation
IGEV copied to clipboard

Considerations with Colmap data

Open KevinCain opened this issue 6 months ago • 0 comments

Since all DTU scenes share nearly identical intrinsics and the robotic camera arm targeted the same camera poses in each scene yielding the nearly identical extrinsics, does IGEV-MVS training on DTU give scale sensitivity and intrinsic bias?

I'm using 'colmap_input.py' to convert to IterMVS-Net form for IGEV-MVS, which works when starting with DTU input images but yields poor depth maps and very low final mask percentages for custom images, e..g:

processing X://dtu//scan400, ref-view00, geo_mask:0.001213 final_mask: 0.001213

As above, is it possible that extrinsics scale could be responsible? In general is there some reference for how the IGEV-MDS DTU model generalizes to custom photo input? Models trained on datasets with limited diversity for camera parameters (including focal length, and principal point) can perform poorly when tested on data with different parameters, a known challenge.

One note: For IGEV-MVS, ‘pair.txt’ must have exactly (10) source images for each reference image. However, the IterMVS-to-Colmap script 'colmap_input.py' exports a source image for every reference image. Therefore I cull the results to exactly (10) source images as per the format specification.

KevinCain avatar Jan 22 '24 03:01 KevinCain