Custom Large-Scale Dataset training with ADMM

[yur1xfd]

Thank you very much for your excellent work and the released code! I have several questions about training on large-scale custom dataset (I've highlighted them in bold). I tried to train my own custom large-scale scene (in COLMAP format) using scripts provided in admm branch. To do that I create my custom config file based mostly on urban3d_admm.yaml config file (just changing name, root_dir and scene parameters and setting use_manhattan_world to False). Then I run large-scale dataset preprocessing script as follows:

cd scripts/preprocess
./preprocess_large_scale_data.sh 0 custom_admm gaussian_splatting

So my custom scene separated into 4 blocks. Then I run the following script to train my separated scene:

cd scripts/train
./train_nvs.sh 0 church custom_admm gaussian_splatting

Since the my config file name contains the word admm train_nvs.sh script try to use conerf.trainers.admm_gaussian_trainer for scene training, but directory /conerf/trainers/ doesn't contains such file and I got an error:

No module named conerf.trainers.admm_gaussian_trainer

So, is this a mistake in the train_nvs.sh script or this trainer will be released soon? Following your README.md the code for ADMM Gaussian Splatting trainer has been already released.

Trying to avoid this error I change name of my custom config file to custom.yaml (in this case /train_nvs.sh script run python train.py), but I faced with issue that GS model start to train only on the first block (with index 0) of my scene. So what should I do to correctly train my custom large-scale scene using admm method? Do I understand correctly that if do not split the scene into blocks and set admm.enable to False, then the vanilla GS will be trained?

[AIBluefisher]

Hi, the train_nvs.sh should not be used to train 3DGS distributedly. It seems I forgot to update this script since it no longer supports training 3DGS distributedly on a single machine. Please follow the instructions to use train_admm_*.sh for distributed training.

[yur1xfd]

Ok, I got it, thank you! And am I understanding this correctly, that the train_nvs.sh script, with the corresponding urban 3d.yaml config, provides the ability to train just a vanilla 3DGS model without any improvements?

[AIBluefisher]

Yes! You can check the difference between urban3d.yaml and urban3d_admm.yaml, for which the latter config file specified some parameters to enable ADMM training. Note that our reimplementation of 3DGS is slightly different from the official implementation and the version of 3DGS we tested during paper submission. But that would not cause much difference in the final results. We will also support more features in the future (e.g. Sky modeling to eliminate floaters caused by sky, etc).

[yur1xfd]

Thanks a lot for your clarifications!