question regarding ns-process-data

[Aru321]

Thank you for your project. While attempting to construct a dataset from a video using ns-process-data, I encountered a challenge regarding the match rate of poses and images as reported by COLMAP. The match rate appears to be significantly low, with only 0.76% of the images yielding poses.

The specific message is as follows: "COLMAP only found poses for 0.76% of the images. This is low. video_to_nerfstudio_dataset.py:145."

Given that I captured the video around the object using my phone, I'm wondering if there are criteria or examples of videos that would lead to an acceptable dataset. Any guidance or examples you can provide would be greatly appreciated.

[abrahamezzeddine]

Most likely you have images that are blurry and lost connection between the images. Did you view the frames that was created via this pipeline if there are lots of blurry images?

Usually, sequential frames provide really good matching if it comes from video.

[Aru321]

Most likely you have images that are blurry and lost connection between the images. Did you view the frames that was created via this pipeline if there are lots of blurry images?

Usually, sequential frames provide really good matching if it comes from video.

Thanks for your reply. I tried directly using COLMAP gui to extract features and match. The result appeared to be great, which made me confused. Maybe something goes wrong with the arguments of ns-process-data?

[abrahamezzeddine]

What arguments are you using in nerfstudio and how are you generating the data in COLMAP? Via automatic reconstruction?

[Aru321]

In nerfstudio, I simply used "ns-process-data video --data xxx --output-dir xxx" following the official instruction. For COLMAP gui, I generated data through "feature extraction -> feature matching -> start reconstruction". I set the camera model as "SIMPLE PINHOLE", does this affect the results?

[aerialtreework]

Try adding the --num-frames-target flag and using less frames for the data. Sometimes, too many frames actually messes with the structure-from-motion algorithm, because there is not enough change in angle between individual images. If you're just doing one full circle around the teddy bear, try this: ns-process-data video --num-frames-target 100 --data xxx --output-dir xxx . You could also try it at 50 and 75 frames.

The advice to look at your images to see if they are blurry is also good. Even though you can use Colmap's GUI to get a result, you should still be able to get something via ns-process-data with basic objects, and the fact that you couldn't typically means that something with the video was less than optimal. It could even be a really flat background or one that is too busy. If your image quality is fine and your background is good, the very next thing I would look at is the number of images that are being run through Colmap, like I mentioned above.

[AntonioMacaronio]

Were poses only found for 2 images? I believe this is an issue with the version of COLMAP.

If it was the case that our 0.76% was only 2 poses, check if you version of COLMAP is 3.8; if it's new that it could be problematic

[andygikling]

Bump - thank you @AntonioMacaronio I was using COLMAP 3.10 which is what came in as the latest from the setup instructions. When I ran this, my image sate of 340 images only matched 2 images as you stated.

When I changed to 3.8 with

conda install -c conda-forge colmap=3.8

And ran it again, I see 335 of 340 images matched. @Aru321 try going backward on the colmap version.