autonomousvision
Custom datasets do not have the same files as the sample datasets
Hello,
I have some general questions regarding the sdfstudio-data format and using custom data. I have converted several custom datasets into sdfstudio format and am having trouble getting results that look anywhere near as good as the models trained using the sample datasets such as dtu-scan65 (skull). I noticed that the dtu-scan65 data contains some files that the custom data does not after using the provided scripts to convert the custom data into sdfstudio format. The following files are in the original dtu-scan65 dataset but not in my custom datasets after converting them to sdfstudio format.
- cameras.npz
- pairs.txt
- sparse_sfm_points.txt
- each image has its own xxx_sfm_points.txt file
In order to test if these files are related to my poor results I took the original RGB images from the dtu-scan65 dataset and converted them back into sdfstudio format according to the same steps I use for other custom datasets, then trained a model and got similarly poor results. Below I've detailed the steps I take for converting custom datasets to sdfstudio format, as well as the results of training a VolSDF model on the original dtu-scan65 dataset, and the one converted from scratch as a "custom" dataset. In the custom version it appears that the camera positions are being projected onto the background..
-
ns-process-data images ....
-
python scripts/datasets/process_nerfstudio_to_sdfstudio.py --data {processed_data} --output-dir {output_dir} --scene-type object --data-type colmap --mono-prior --omnidata-path /omnidata/omnidata_tools/torch/ --pretrained-models omnidata/pretrained_models/
-
ns-train volsdf --pipeline.model.sdf-field.inside-outside True --output-dir {output_dir} --data {sdfstudio_dataset}
-
ns-export tsdf --load-config {config.yml} --output-dir {output_dir} --target-num-faces 100000 --num-pixels-per-side 2048 --use-bounding-box True --bounding-box-min -0.5 -0.5 -0.5 --bounding-box-max 0.5 0.5 0.5
Sample dtu-scan65 Dataset Results:
"Custom" Dataset using the (same images but running through the conversion pipeline):
Can anyone explain what the files mentioned above are, how they are used, how they are created, and why they are not created when I convert custom datasets? And are the absence of any of these files the reason my results are so poor? Thanks so much.
Some files in the original dtu-scan65 dataset are not necessary for the method you want to use. Different NeRR methods have different data input requirements. For example, the pairs.txt file you mentioned is only applicable to the geo-neus series of methods, because this series of methods requires NCC calculation.
